Skip to content

Commit ed8016d

Browse files
committed
fix(parser-wikipedia): add valid User-Agent to respect robot policy
1 parent 031f18b commit ed8016d

1 file changed

Lines changed: 2 additions & 1 deletion

File tree

‎html_parsing/wikipedia/common.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,9 +19,10 @@ def request(self, *args, **kwargs):
1919
session = TimeoutRequestsSession()
2020
session.headers.update(
2121
{
22-
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:121.0) Gecko/20100101 Firefox/121.0",
22+
"User-Agent": "WikiEpisodesParser/1.0 (https://github.com/gil9red/SimplePyScripts; Owner: gil9red)",
2323
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8",
2424
"Accept-Language": "ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3",
25+
"Accept-Encoding": "gzip",
2526
"Connection": "keep-alive",
2627
"Upgrade-Insecure-Requests": "1",
2728
}

0 commit comments

Comments
 (0)