Skip to content

Commit 02f185f

Browse files
committed
chore!: drop Python 3.10 support (#2295)
First cleanup PR for v2. Python 3.10 reached end of life on 2026-10-04, so Crawlee v2 requires Python 3.11 or newer. - Bumps `requires-python` to `>=3.11` in the package and in the project template, updates the classifiers, the ty target and the CI matrix, and regenerates `uv.lock`. - Applies the ruff fixes the new target enables: `datetime.UTC`, builtin `TimeoutError`, and `typing` imports of `Self` / `NotRequired`. `Event` becomes a `StrEnum`, so `str(Event.X)` returns the value. - Removes the 3.11 skips and the `_restore_crawlers_binding` fixture, which only existed for 3.10. - Adds an "Upgrading to v2" page and updates the Python version in the README and docs. The `apify_client<3.0.0` dev pin and the `async-timeout` dependency (#2052) will go in separate PRs. Closes: #1721 *✍️ Drafted by Claude Code*
1 parent 508f544 commit 02f185f

93 files changed

Lines changed: 328 additions & 1502 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/_checks.yaml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -41,13 +41,13 @@ jobs:
4141
name: Lint check
4242
uses: apify/workflows/.github/workflows/python_lint_check.yaml@main
4343
with:
44-
python_versions: '["3.10", "3.11", "3.12", "3.13", "3.14", "3.15"]'
44+
python_versions: '["3.11", "3.12", "3.13", "3.14", "3.15"]'
4545

4646
type_check:
4747
name: Type check
4848
uses: apify/workflows/.github/workflows/python_type_check.yaml@main
4949
with:
50-
python_versions: '["3.10", "3.11", "3.12", "3.13", "3.14", "3.15"]'
50+
python_versions: '["3.11", "3.12", "3.13", "3.14", "3.15"]'
5151

5252
markdown_lint_check:
5353
name: Markdown lint check
@@ -135,7 +135,7 @@ jobs:
135135
uses: apify/workflows/.github/workflows/python_unit_tests.yaml@main
136136
secrets: inherit
137137
with:
138-
python_versions: '["3.10", "3.11", "3.12", "3.13", "3.14", "3.15"]'
138+
python_versions: '["3.11", "3.12", "3.13", "3.14", "3.15"]'
139139
operating_systems: '["ubuntu-latest", "windows-latest", "macos-latest"]'
140140
python_version_for_codecov: "3.15"
141141
operating_system_for_codecov: ubuntu-latest

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616
<p align="center">
1717
<a href="https://badge.fury.io/py/crawlee" rel="nofollow"><img src="https://badge.fury.io/py/crawlee.svg" alt="PyPI package version"></a>
1818
<a href="https://pypi.org/project/crawlee/" rel="nofollow"><img src="https://img.shields.io/pypi/dm/crawlee" alt="PyPI package downloads"></a>
19-
<a href="https://pypi.org/project/crawlee/" rel="nofollow"><img src="https://img.shields.io/badge/python-3.10%2B-blue" alt="PyPI Python version"></a>
19+
<a href="https://pypi.org/project/crawlee/" rel="nofollow"><img src="https://img.shields.io/badge/python-3.11%2B-blue" alt="PyPI Python version"></a>
2020
<a href="https://github.com/apify/crawlee-python/actions/workflows/on_master.yaml"><img src="https://github.com/apify/crawlee-python/actions/workflows/on_master.yaml/badge.svg?branch=master" alt="Build status"></a>
2121
<a href="https://codecov.io/gh/apify/crawlee-python"><img src="https://codecov.io/gh/apify/crawlee-python/graph/badge.svg?token=cCju61iPQG" alt="Codecov report"></a>
2222
<a href="https://github.com/apify/crawlee-python/blob/master/LICENSE"><img src="https://img.shields.io/pypi/l/crawlee" alt="License"></a>

‎docs/guides/code_examples/http_crawlers/selectolax_context.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
from dataclasses import dataclass, fields
2+
from typing import Self
23

34
from selectolax.lexbor import LexborHTMLParser
4-
from typing_extensions import Self
55

66
from crawlee.crawlers._abstract_http import ParsedHttpCrawlingContext
77

‎docs/introduction/01_setting_up.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ This guide will help you get started with Crawlee by setting it up on your compu
1414

1515
Before installing Crawlee itself, make sure that your system meets the following requirements:
1616

17-
- **Python 3.10 or higher**: Crawlee requires Python 3.10 or a newer version. You can download Python from the [official website](https://python.org/downloads/).
17+
- **Python 3.11 or higher**: Crawlee requires Python 3.11 or a newer version. You can download Python from the [official website](https://python.org/downloads/).
1818
- **Python package manager**: While this guide uses [pip](https://pip.pypa.io/) (the most common package manager), you can also use any package manager you want. You can download pip from the [official website](https://pip.pypa.io/en/stable/installation/).
1919

2020
### Verifying prerequisites

‎docs/quick-start/index.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ Crawlee offers the following main crawler classes: <ApiLink to="class/BeautifulS
2323

2424
:::caution Minimum Python version
2525

26-
Crawlee requires Python 3.10 or higher.
26+
Crawlee requires Python 3.11 or higher.
2727

2828
:::
2929

‎docs/upgrading/upgrading_to_v2.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
id: upgrading-to-v2
3+
title: Upgrading to v2
4+
---
5+
6+
This page summarizes the breaking changes between Crawlee for Python v1 and v2.
7+
8+
## Python version support
9+
10+
We drop support for Python 3.10, which reached its end of life in October 2026. The minimum supported version is now Python 3.11.
11+
12+
## `Event` is a `StrEnum`
13+
14+
`Event` now subclasses `enum.StrEnum`. Calling `str()` on a member or formatting it in an f-string returns its value, for example `'systemInfo'`, where v1 returned `'Event.SYSTEM_INFO'`. Comparisons with plain strings work the same way as in v1.

‎pyproject.toml‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,13 +10,12 @@ authors = [{ name = "Apify Technologies s.r.o.", email = "support@apify.com" }]
1010
license = "Apache-2.0"
1111
license-files = ["LICENSE"]
1212
readme = "README.md"
13-
requires-python = ">=3.10"
13+
requires-python = ">=3.11"
1414
classifiers = [
1515
"Development Status :: 5 - Production/Stable",
1616
"Environment :: Console",
1717
"Intended Audience :: Developers",
1818
"Operating System :: OS Independent",
19-
"Programming Language :: Python :: 3.10",
2019
"Programming Language :: Python :: 3.11",
2120
"Programming Language :: Python :: 3.12",
2221
"Programming Language :: Python :: 3.13",
@@ -289,7 +288,7 @@ filterwarnings = [
289288
]
290289

291290
[tool.ty.environment]
292-
python-version = "3.10"
291+
python-version = "3.11"
293292

294293
[tool.ty.src]
295294
include = ["src", "tests", "scripts", "docs", "website"]

‎src/crawlee/_autoscaling/_types.py‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
from __future__ import annotations
22

33
from dataclasses import dataclass, field
4-
from datetime import datetime, timedelta, timezone
4+
from datetime import UTC, datetime, timedelta
55
from typing import TYPE_CHECKING, Annotated
66

77
from pydantic import Field
@@ -46,7 +46,7 @@ class SystemInfo:
4646
client_info: LoadRatioInfo
4747
"""The client load ratio."""
4848

49-
created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
49+
created_at: datetime = field(default_factory=lambda: datetime.now(UTC))
5050
"""The time at which the system load information was measured."""
5151

5252
@property
@@ -80,7 +80,7 @@ class CpuSnapshot:
8080
max_used_ratio: float
8181
"""The maximum ratio of CPU that is considered acceptable."""
8282

83-
created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
83+
created_at: datetime = field(default_factory=lambda: datetime.now(UTC))
8484
"""The time at which the system load information was measured."""
8585

8686
@property
@@ -108,7 +108,7 @@ class MemorySnapshot:
108108
max_used_memory_ratio: float
109109
"""The maximum acceptable ratio of `current_size` to `max_memory_size`."""
110110

111-
created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
111+
created_at: datetime = field(default_factory=lambda: datetime.now(UTC))
112112
"""The time at which the system load information was measured."""
113113

114114
@property
@@ -132,7 +132,7 @@ class EventLoopSnapshot:
132132
max_delay: timedelta
133133
"""The maximum delay that is considered acceptable."""
134134

135-
created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
135+
created_at: datetime = field(default_factory=lambda: datetime.now(UTC))
136136
"""The time at which the system load information was measured."""
137137

138138
@property
@@ -159,7 +159,7 @@ class ClientSnapshot:
159159
max_error_count: int
160160
"""The maximum number of errors that is considered acceptable."""
161161

162-
created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
162+
created_at: datetime = field(default_factory=lambda: datetime.now(UTC))
163163
"""The time at which the system load information was measured."""
164164

165165
@property

‎src/crawlee/_autoscaling/autoscaled_pool.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -243,7 +243,7 @@ async def _worker_task_orchestrator(self, run: _AutoscaledPoolRun) -> None:
243243

244244
continue
245245

246-
with suppress(asyncio.TimeoutError):
246+
with suppress(TimeoutError):
247247
await asyncio.wait_for(run.worker_tasks_updated.wait(), timeout=0.5)
248248
except Exception as exc:
249249
# Surface the error through `run.result` only once the cleanup below has awaited the worker tasks,
@@ -290,7 +290,7 @@ async def _worker_task(self) -> None:
290290
self._run_task_function(),
291291
timeout=self._TASK_TIMEOUT.total_seconds() if self._TASK_TIMEOUT is not None else None,
292292
)
293-
except asyncio.TimeoutError:
293+
except TimeoutError:
294294
if self._TASK_TIMEOUT is None:
295295
# Without a timeout, `wait_for` cannot time out - the error comes from the task function itself.
296296
raise

‎src/crawlee/_autoscaling/snapshotter.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
from __future__ import annotations
44

55
from bisect import insort
6-
from datetime import datetime, timedelta, timezone
6+
from datetime import UTC, datetime, timedelta
77
from logging import WARNING, getLogger
88
from typing import TYPE_CHECKING, TypeVar, cast
99

@@ -107,7 +107,7 @@ def __init__(
107107
self._snapshot_event_loop_task = RecurringTask(self._snapshot_event_loop, self._EVENT_LOOP_SNAPSHOT_INTERVAL)
108108
self._snapshot_client_task = RecurringTask(self._snapshot_client, self._CLIENT_SNAPSHOT_INTERVAL)
109109

110-
self._timestamp_of_last_memory_warning: datetime = datetime.now(timezone.utc) - timedelta(hours=1)
110+
self._timestamp_of_last_memory_warning: datetime = datetime.now(UTC) - timedelta(hours=1)
111111

112112
# Flag to indicate the context state.
113113
self._active = False

0 commit comments

Comments
 (0)