Skip to content

Commit ccfc7ab

Browse files
test(uptime): Mock the robots.txt fetch in candidate URL tests (#127947)
Four `ProcessCandidateUrlTest` tests (`test_succeeds_*`) call `process_candidate_url` for random `https://sentry-<hex>.io` domains. That fetches `robots.txt` over the real network. The outbound network guard in report mode (#127944) caught these on a full backend run. These domains don't resolve, so `check_url_robots_txt` catches the error and allows the URL. The class now has an autouse `responses` fixture that answers those `robots.txt` URLs with a 404. `get_robots_txt_parser` turns that into an empty, allow-all parser, so the tests get the same outcome without a DNS lookup or a connection. Tests that already patch `get_robots_txt_parser` don't change. No assertions change. This PR only changes tests and doesn't depend on #127944. I ran the file locally with `SENTRY_NETWORK_GUARD=strict`: it passes, and the guard records no outbound attempts. <!-- junior-request-attribution:start --> via @joshuarli. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:agent%3A35c0cbe32045897a494787cce713d692 --> -- [View Junior Session](https://junior-prod.sentry.dev/conversations/agent%3A35c0cbe32045897a494787cce713d692) [[Sentry]](https://sentry.sentry.io/explore/conversations/agent%3A35c0cbe32045897a494787cce713d692/?project=4510944073809921) <!-- junior-session-footer:end --> Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Joshua Li <14209781+joshuarli@users.noreply.github.com>
1 parent 41f9e5a commit ccfc7ab

1 file changed

Lines changed: 14 additions & 0 deletions

File tree

‎tests/sentry/uptime/autodetect/test_tasks.py‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,15 @@
11
from __future__ import annotations
22

3+
import re
34
import uuid
5+
from collections.abc import Generator
46
from datetime import datetime, timedelta
57
from unittest import mock
68
from unittest.mock import call
79
from urllib.robotparser import RobotFileParser
810

11+
import pytest
12+
import responses
913
from django.utils import timezone
1014

1115
from sentry.deletions.tasks.scheduled import run_scheduled_deletions
@@ -227,6 +231,16 @@ def test_should_not_detect(self) -> None:
227231

228232
@freeze_time()
229233
class ProcessCandidateUrlTest(UptimeTestCase):
234+
@pytest.fixture(autouse=True)
235+
def mock_robots_txt_not_found(self) -> Generator[None]:
236+
# The generated test domains don't exist, so give them no robots.txt
237+
# rather than letting the fetch go out to the network.
238+
with responses.RequestsMock(assert_all_requests_are_fired=False) as rsps:
239+
rsps.add(
240+
responses.GET, re.compile(r"https://sentry-[0-9a-f]{8}\.io/robots\.txt"), status=404
241+
)
242+
yield
243+
230244
@with_feature("organizations:uptime")
231245
def test_succeeds_new(self) -> None:
232246
url = make_unique_test_url()

0 commit comments

Comments
 (0)