Skip to content

Commit d8ec583

Browse files
AdrianAtZyteAdrianAtZyte
authored
Add benchmarks and regenerate the parser comparison from measurements (#83)
* Add benchmarks and regenerate the parser comparison from measurements * Fix CodSpeed support
1 parent 4ec7701 commit d8ec583

14 files changed

Lines changed: 706 additions & 31 deletions

‎.github/dependabot.yml‎

Lines changed: 17 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,19 @@
11
version: 2
22
updates:
3-
- package-ecosystem: github-actions
4-
directory: "/"
5-
schedule:
6-
interval: monthly
7-
groups:
8-
github-actions:
9-
patterns:
10-
- "*"
11-
cooldown:
12-
default-days: 7
3+
- package-ecosystem: pip
4+
directory: /benchmarks
5+
schedule:
6+
interval: monthly
7+
cooldown:
8+
default-days: 7
9+
10+
- package-ecosystem: github-actions
11+
directory: /
12+
schedule:
13+
interval: monthly
14+
groups:
15+
github-actions:
16+
patterns:
17+
- "*"
18+
cooldown:
19+
default-days: 7

‎.github/workflows/benchmarks.yml‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
name: Benchmarks
2+
on:
3+
push:
4+
branches: [master]
5+
pull_request:
6+
paths:
7+
- src/**
8+
- benchmarks/**
9+
- .github/workflows/benchmarks.yml
10+
- tox.ini
11+
workflow_dispatch:
12+
13+
concurrency:
14+
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
15+
cancel-in-progress: true
16+
17+
permissions: {}
18+
19+
jobs:
20+
codspeed:
21+
runs-on: ubuntu-latest
22+
permissions:
23+
contents: read
24+
id-token: write # OIDC authentication with CodSpeed
25+
steps:
26+
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
27+
with:
28+
persist-credentials: false
29+
30+
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
31+
with:
32+
python-version: "3.14.7"
33+
34+
- name: Install
35+
run: |
36+
pip install -U tox
37+
38+
- uses: CodSpeedHQ/action@0ca9cbbf4623b599a6c3ed4fc8a922942705d9f1 # v5.0.2
39+
with:
40+
mode: simulation
41+
exclude-allocations: true
42+
run: tox -e benchmarks

‎.github/workflows/comparison.yml‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
name: Comparison table
2+
on:
3+
workflow_dispatch:
4+
push:
5+
branches: [master]
6+
paths:
7+
- src/**
8+
- benchmarks/**
9+
10+
permissions:
11+
contents: write
12+
pull-requests: write
13+
14+
jobs:
15+
comparison:
16+
runs-on: ubuntu-latest
17+
steps:
18+
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
19+
with:
20+
persist-credentials: false
21+
22+
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
23+
with:
24+
python-version: "3.14.7"
25+
26+
- name: Install
27+
run: |
28+
sudo apt-get update && sudo apt-get install -y valgrind
29+
pip install -U tox
30+
31+
- name: Regenerate the table
32+
run: tox -e comparison -- --write
33+
34+
- uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1
35+
with:
36+
branch: comparison-table
37+
title: Update the parser comparison table
38+
commit-message: Update the parser comparison table
39+
body: |
40+
Regenerated by `benchmarks/compare.py` from commit ${{ github.sha }}.
41+
delete-branch: true

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,3 +21,6 @@ dist/
2121
.coverage
2222
.coverage.*
2323
coverage.*
24+
25+
# test reports
26+
testenv.junit.xml

‎README.rst‎

Lines changed: 73 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -86,27 +86,79 @@ Comparison
8686
==========
8787

8888
The following table compares Protego to the most popular ``robots.txt`` parsers
89-
implemented in Python or featuring Python bindings:
90-
91-
+----------------------------+---------+-----------------+--------+---------------------------+
92-
| | Protego | RobotFileParser | Reppy | Robotexclusionrulesparser |
93-
+============================+=========+=================+========+===========================+
94-
| Implementation language | Python | Python | C++ | Python |
95-
+----------------------------+---------+-----------------+--------+---------------------------+
96-
| Reference specification | Google_ | `Martijn Koster’s 1996 draft`_ |
97-
+----------------------------+---------+-----------------+--------+---------------------------+
98-
| `Wildcard support`_ | ✓ | | ✓ | ✓ |
99-
+----------------------------+---------+-----------------+--------+---------------------------+
100-
| `Length-based precedence`_ | ✓ | | ✓ | |
101-
+----------------------------+---------+-----------------+--------+---------------------------+
102-
| Performance_ | | +40% | +1300% | -25% |
103-
+----------------------------+---------+-----------------+--------+---------------------------+
104-
105-
.. _Google: https://developers.google.com/search/reference/robots_txt
106-
.. _Length-based precedence: https://developers.google.com/search/reference/robots_txt#order-of-precedence-for-group-member-lines
107-
.. _Martijn Koster’s 1996 draft: https://www.robotstxt.org/norobots-rfc.txt
108-
.. _Performance: https://anubhavp28.github.io/gsoc-weekly-checkin-12/
109-
.. _Wildcard support: https://developers.google.com/search/reference/robots_txt#url-matching-based-on-path-values
89+
implemented in Python. Performance is the speed difference against Protego,
90+
measured as executed instructions, so a positive value means faster than
91+
Protego.
92+
93+
.. comparison-table-start
94+
95+
.. list-table::
96+
:header-rows: 1
97+
:stub-columns: 1
98+
99+
* -
100+
- Protego
101+
- RobotFileParser
102+
- robotspy
103+
- Robotexclusionrulesparser
104+
* - Version tested
105+
-
106+
- Python 3.14.7
107+
- 0.13.0
108+
- 1.7.1
109+
* - Reference specification
110+
- Google_
111+
- `Martijn Koster's 1996 draft`_
112+
- `RFC 9309`_
113+
- `Martijn Koster's 1996 draft`_
114+
* - `Wildcard support`_
115+
- ✓
116+
-
117+
- ✓
118+
- ✓
119+
* - `Length-based precedence`_
120+
- ✓
121+
-
122+
- ✓
123+
-
124+
* - Crawl-delay
125+
- ✓
126+
- ✓
127+
-
128+
-
129+
* - Request-rate
130+
- ✓
131+
-
132+
-
133+
-
134+
* - Visit-time
135+
- ✓
136+
-
137+
-
138+
-
139+
* - Sitemaps
140+
- ✓
141+
- ✓
142+
- ✓
143+
- ✓
144+
* - Host
145+
- ✓
146+
-
147+
-
148+
-
149+
* - Performance
150+
-
151+
- +40%
152+
-
153+
- -25%
154+
155+
.. comparison-table-end
156+
157+
.. _Google: https://developers.google.com/crawling/docs/robots-txt/robots-txt-spec
158+
.. _Length-based precedence: https://developers.google.com/crawling/docs/robots-txt/robots-txt-spec#order-of-precedence-for-rules
159+
.. _Martijn Koster's 1996 draft: https://www.robotstxt.org/norobots-rfc.txt
160+
.. _RFC 9309: https://www.rfc-editor.org/rfc/rfc9309
161+
.. _Wildcard support: https://developers.google.com/crawling/docs/robots-txt/robots-txt-spec#url-matching-based-on-path-values
110162

111163

112164
API Reference

‎benchmarks/__init__.py‎

Whitespace-only changes.

0 commit comments

Comments
 (0)