Skip to content

Commit 9a8a89d

Browse files
docs(backend): Refresh backend agent and subsystem docs against code (#127884)
This is a docs-refresh pass over the backend developer docs. I checked each claim in `AGENTS.md`, `src/AGENTS.md`, `tests/AGENTS.md`, and the backend subsystem docs against the current code. Wrong claims are fixed, and text that repeats something a lint, a runtime check, or another doc already owns is cut. Frontend docs are out of scope. Fixes that change what an agent or a human would actually do: - **Tasks:** the guide described Celery (`queue`, `max_retries`). Tasks run on taskbroker now, so it covers `namespace`, `retry=Retry(...)`, and `TASKWORKER_IMPORTS` instead. - **Silos:** `RegionSiloEndpoint` and `src/sentry/models/outbox.py` don't exist. The guide now points to the `@cell_silo_model`/`@control_silo_model` and endpoint decorators. It also said org management lives in control, but `Organization` is a cell-silo model. - **Linting:** `prek run ruff` fails with "No hooks found". The hook id is `ruff-check`. - **Integrations:** there is no `integrations/registry.py`. Providers register through `SENTRY_DEFAULT_INTEGRATIONS`. - **Snuba CI:** the doc said Snuba CI runs `tests/snuba/`. It actually picks tests by the `snuba_ci`/`snuba` markers. - **Anti-patterns:** I removed "WRONG: direct model import in API". About 135 of 158 files in `api/endpoints` do exactly that. - **Backup tests:** the README described a `JSONMutatingComparator`/`COMPARATORS` sentinel scheme that no longer exists, and named the wrong snapshot file. It now describes `JSONScrubbingComparator`, and it holds the add-a-model checklist that used to sit in `tests/AGENTS.md`. - **Subsystem docs:** I fixed moved paths for integration parsers and the proxy endpoint, plus workflow engine references to the deleted `migration_helpers/` and `rules/history/` modules. I kept the existing layout (area `AGENTS.md` files, subsystem docs next to code, skills for workflows) and did not add a top-level `docs/` tree. Decisions for reviewers: - The "import at top of module" and `hasattr`→`isinstance` rules could be lint rules (`PLC0415`, or a custom check). If one gets enabled, its prose line can be deleted. - `.agents/skills/backend-conventions` mentions tracing helpers. I didn't edit that skill here. <!-- junior-request-attribution:start --> via @joshuarli. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:slack%3ACUHS29QJ0%3A1792184857.666969 --> -- [View Junior Session](https://junior-prod.sentry.dev/conversations/slack%3ACUHS29QJ0%3A1792184857.666969) [[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3ACUHS29QJ0%3A1792184857.666969/?project=4510944073809921) <!-- junior-session-footer:end --> Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Joshua Li <14209781+joshuarli@users.noreply.github.com>
1 parent 37f41f0 commit 9a8a89d

8 files changed

Lines changed: 80 additions & 127 deletions

File tree

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ prek is the single entrypoint for all lint, format, and type-checking tools. Bef
4545

4646
```bash
4747
SENTRY_MYPY_PRE_PUSH=1 .venv/bin/prek run -q mypy --files src/sentry/foo/bar.py --stage pre-push
48-
.venv/bin/prek run -q ruff --files src/sentry/foo/bar.py
48+
.venv/bin/prek run -q ruff-check --files src/sentry/foo/bar.py
4949
```
5050

5151
If a hook fails, fix the issues, stage changes, then re-run until it passes.

‎src/AGENTS.md‎

Lines changed: 41 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -1,47 +1,42 @@
11
# Backend Development Guide
22

3-
> For critical commands, see the "Command Execution Guide" section in `/AGENTS.md` in the repository root.
3+
> For setup, lint, and test commands, see the "Command Execution Guide" section in `/AGENTS.md` in the repository root.
44
55
## Security Guidelines
66

77
### Preventing Indirect Object References (IDOR)
88

9-
Multi-tenant queries MUST be scoped — never trust user-supplied IDs alone.
9+
Multi-tenant queries MUST be scoped. Never trust user-supplied IDs alone.
1010

11-
- Always filter by `organization_id` and/or `project_id`:
11+
- Always filter by `organization_id` or `project_id`:
1212
`Resource.objects.get(id=..., organization_id=organization.id)`.
13-
- Never read `request.data["project_id"]` / `request.GET["project_id"]` directly. Use `self.get_projects(request=request, organization=organization, project_ids=...)`, which validates permissions.
13+
- Never read `request.data["project_id"]` or `request.GET["project_id"]` directly. Use `self.get_projects(request=request, organization=organization, project_ids=...)`, which checks permissions.
1414

1515
For access-control review or deeper authorization work, use the **`django-access-review`** and **`sentry-security`** skills.
1616

1717
## Development Services
1818

19-
Local dependencies are managed by `devservices` (config: `devservices/config.yml`).
20-
21-
📖 Full devservices documentation: https://develop.sentry.dev/development-infrastructure/devservices.md
19+
`devservices` runs local dependencies from `devservices/config.yml`. Full documentation: https://develop.sentry.dev/development-infrastructure/devservices/
2220

2321
## AI Assistant Quick Decision Trees
2422

2523
### "User wants to add an API endpoint"
2624

27-
1. Check if endpoint already exists: `grep -r "endpoint_name" src/sentry/api/`
28-
2. Inherit from appropriate base:
29-
- Organization-scoped: `OrganizationEndpoint`
30-
- Project-scoped: `ProjectEndpoint`
31-
- Region silo: `RegionSiloEndpoint`
32-
3. File locations:
33-
- Endpoint: `src/sentry/api/endpoints/{resource}.py`
34-
- URL: `src/sentry/api/urls.py`
35-
- Test: `tests/sentry/api/endpoints/test_{resource}.py`
36-
- Serializer: `src/sentry/api/serializers/models/{model}.py`
37-
4. Document with drf-spectacular decorators → use the **`document-api-endpoint`** skill.
38-
39-
### "User wants to add a Celery task"
40-
41-
1. Location: `src/sentry/tasks/{category}.py`
42-
2. Use `@instrumented_task` decorator
43-
3. Set appropriate `queue` and `max_retries`
44-
4. Test location: `tests/sentry/tasks/test_{category}.py`
25+
1. Check whether the endpoint already exists: `grep -rn "endpoint_name" src/sentry/`.
26+
2. Inherit from the scope base: `OrganizationEndpoint` (`src/sentry/api/bases/organization.py`) or `ProjectEndpoint` (`src/sentry/api/bases/project.py`).
27+
3. Decorate the class with `@cell_silo_endpoint` or `@control_silo_endpoint` from `src/sentry/api/base.py`.
28+
4. Put the endpoint in the owning domain's `endpoints/` package (for example `src/sentry/core/endpoints/`) or in `src/sentry/api/endpoints/`. Register the route in `src/sentry/api/urls.py`, or in the domain `urls.py` it includes. The test file mirrors the endpoint path under `tests/`.
29+
5. Document it with drf-spectacular decorators → use the **`document-api-endpoint`** skill.
30+
31+
### "User wants to add a task"
32+
33+
Tasks run on taskbroker, not Celery.
34+
35+
1. Decorate the function with `@instrumented_task` from `src/sentry/tasks/base.py`. Pass `name`, a `namespace` from `src/sentry/taskworker/namespaces.py`, and `retry=Retry(...)` from `taskbroker_client.retry` when the task retries.
36+
2. Add a new task module to `TASKWORKER_IMPORTS` in `src/sentry/conf/server.py`. `tests/sentry/taskworker/test_config.py` fails when a listed path does not import.
37+
3. The test file mirrors the module path under `tests/`.
38+
39+
Full documentation: https://develop.sentry.dev/backend/application-domains/tasks/
4540

4641
### Serializers: Avoiding N+1 Queries
4742

@@ -51,30 +46,25 @@ NEVER query the database in `serialize()` for bulk requests. `serialize()` runs
5146

5247
### API Documentation
5348

54-
- OpenAPI spec generation: `make build-api-docs`
55-
- API ownership tracked in `src/sentry/apidocs/api_ownership_allowlist_dont_modify.py`
56-
- Documenting/typing an endpoint → use the **`document-api-endpoint`** skill.
49+
- Generate the OpenAPI spec with `make build-api-docs`.
50+
- API ownership is tracked in `src/sentry/apidocs/api_ownership_allowlist_dont_modify.py`.
5751

5852
### API Design Rules
5953

6054
1. Route: `/api/0/organizations/{org}/projects/{project}/`
61-
2. Use `snake_case` for URL params
62-
3. Use `camelCase` for request/response bodies
63-
4. Return strings for numeric IDs
64-
5. Implement pagination with `cursor`
65-
6. Use `GET` for read, `POST` for create, `PUT` for update
66-
7. **Error responses MUST use the `"detail"` key** (Django REST Framework convention) — not `"error"` or `"message"`:
55+
2. Use `snake_case` for URL params.
56+
3. Use `camelCase` for request and response bodies.
57+
4. Return strings for numeric IDs.
58+
5. Paginate with `cursor`.
59+
6. Use `GET` to read, `POST` to create, `PUT` to update.
60+
7. **Error responses MUST use the `"detail"` key**, not `"error"` or `"message"`:
6761
`return Response({"detail": "Invalid input"}, status=400)`.
6862

6963
## Common Patterns
7064

7165
### Viewer/Organization Context
7266

73-
Viewer identity is wired through the app via the `ViewerContext` contextvar; use `sentry.viewer_context.get_viewer_context()` instead of explicitly threading org/user identity when the current viewer is in scope.
74-
75-
### Feature Flags
76-
77-
See the **feature-flags** skill (`.agents/skills/feature-flags/`) for registration, the `features.has(...)` check, and test usage.
67+
The `ViewerContext` contextvar carries the current viewer's identity. When the viewer is in scope, call `sentry.viewer_context.get_viewer_context()` instead of passing org and user identity through arguments.
7868

7969
### Permissions
8070

@@ -90,16 +80,15 @@ class MyPermission(SentryPermission):
9080

9181
### Logging, Tracing, Metrics, Options
9282

93-
Logging (`logger.info/exception`, `LOG005`/`LOG011`), tracing/spans (`sentry.utils.tracing`), metrics tag cardinality, and the options system (`options.get()`) → use the **`backend-conventions`** skill.
83+
Logging (`LOG005`, `LOG011`), spans, metric tag cardinality, and `options.get()` → use the **`backend-conventions`** skill.
9484

9585
## Architecture Rules
9686

9787
### Silo Mode
9888

99-
- **Control Silo**: User auth, billing, organization management
100-
- **Region Silo**: Project data, events, issues
101-
- Check model's silo in `src/sentry/models/outbox.py`; use `@cell_silo_endpoint` or `@control_silo_endpoint`.
102-
- Never join across silos; use `outbox` for cross-silo updates. For silo/RPC/outbox work, use the **`hybrid-cloud-rpc`**, **`hybrid-cloud-outboxes`**, and **`cell-architecture`** skills.
89+
- The control silo holds global data such as `User`, `Integration`, and `OrganizationMapping`. The cell silo holds organization-scoped data such as `Organization`, `Project`, and `Group`.
90+
- Each model declares its silo with `@control_silo_model` or `@cell_silo_model` (`src/sentry/db/models/base.py`). Each endpoint declares it with `@control_silo_endpoint` or `@cell_silo_endpoint`.
91+
- Never join across silos. Use outboxes (`src/sentry/hybridcloud/models/outbox.py`) for cross-silo updates. For silo, RPC, and outbox work, use the **`hybrid-cloud-rpc`**, **`hybrid-cloud-outboxes`**, and **`cell-architecture`** skills.
10392

10493
### Database Guidelines
10594

@@ -110,24 +99,15 @@ Logging (`logger.info/exception`, `LOG005`/`LOG011`), tracing/spans (`sentry.uti
11099

111100
### Redis TTLs
112101

113-
**Every new Redis key sets a TTL, or is registered with Infrastructure Engineering as accepted durable data.** `CommonRedisCache.set` and `RedisKVStorage.set` raise `MissingTTL` rather than write a key with no expiry. There is no opt-out argument: the exemption is granted by Infrastructure Engineering, not at the callsite.
114-
115-
A bare `SET` or `GETSET` over an existing key clears its TTL; `SETEX` replaces it with the supplied expiry. `SADD`, `ZADD`, `HSET`, `HINCRBY` and `INCR` preserve an existing TTL. A TTL refreshed on every write does not bound the key's lifetime — shard by time window and give each shard a fixed TTL instead. Full rules: https://develop.sentry.dev/backend/application-domains/redis/.
102+
**Every new Redis key sets a TTL, or is registered with Infrastructure Engineering as accepted durable data.** `CommonRedisCache.set` and `RedisKVStorage.set` raise `MissingTTL` for a key with no expiry, and there is no opt-out argument. Which commands drop an existing TTL, and why a sliding TTL is not a bound: https://develop.sentry.dev/backend/application-domains/redis/
116103

117104
## Anti-Patterns (NEVER DO)
118105

119106
```python
120-
# WRONG: Direct model import in API
121-
from sentry.models import Organization # NO!
122-
123-
# RIGHT: Use endpoint bases
124-
from sentry.api.bases.organization import OrganizationEndpoint
125-
126-
# WRONG: Synchronous external calls
107+
# WRONG: Slow external calls inside a request
127108
response = requests.get(url) # NO!
128109

129-
# RIGHT: Use Celery task
130-
from sentry.tasks import fetch_external_data
110+
# RIGHT: Move the call into a task (see "User wants to add a task")
131111
fetch_external_data.delay(url)
132112

133113
# WRONG: N+1 queries
@@ -157,18 +137,19 @@ def my_function():
157137

158138
## File Location Map
159139

160-
- **Models**: `src/sentry/models/{model}.py`
161-
- **API Endpoints**: `src/sentry/api/endpoints/{resource}.py`
140+
- **Models**: `src/sentry/models/{model}.py` or the owning domain's `models/` package
141+
- **API Endpoints**: `src/sentry/api/endpoints/{resource}.py` or `src/sentry/{domain}/endpoints/`
162142
- **Serializers**: `src/sentry/api/serializers/models/{model}.py`
163-
- **Tasks**: `src/sentry/tasks/{category}.py`
143+
- **Tasks**: `src/sentry/tasks/{category}.py` or `src/sentry/{domain}/tasks/`
144+
- **Task namespaces**: `src/sentry/taskworker/namespaces.py`
164145
- **Integrations**: `src/sentry/integrations/{provider}/`
165146
- **Permissions**: `src/sentry/api/permissions.py`
166147
- **Feature Flags**: `src/sentry/features/permanent.py` or `temporary.py`
167148
- **Utils**: `src/sentry/utils/{category}.py`
168149

169150
## Integration Development
170151

171-
Add an integration under `src/sentry/integrations/{name}/` with `__init__.py`, `integration.py` (inherit from `Integration`), `client.py` (API client), and `webhooks/` if needed. Register it in `src/sentry/integrations/registry.py` and add a feature flag in `temporary.py`.
152+
Add an integration under `src/sentry/integrations/{name}/`. Subclass `IntegrationProvider` and `IntegrationInstallation` from `src/sentry/integrations/base.py`; `src/sentry/integrations/example/integration.py` is a working reference. Register the provider class path in `SENTRY_DEFAULT_INTEGRATIONS` in `src/sentry/conf/server.py`, and gate it with a flag in `src/sentry/features/temporary.py`.
172153

173154
## Python Typing
174155

‎src/sentry/middleware/integrations/integration_control_middleware.md‎

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,11 @@ Incoming webhooks fired by integration providers notify us when changes occur in
88

99
## How it Works
1010

11-
The magic happens in the [`IntegrationControlMiddleware`](src/sentry/middleware/integrations/integration_control.py). Here, we do the following steps:
11+
The magic happens in the [`IntegrationControlMiddleware`](/src/sentry/middleware/integrations/integration_control.py). Here, we do the following steps:
1212

1313
- If an HTTP request is received to `/extensions/*` (which is the prefix for all our webhooks) it is further inspected. If not, we fall through this middleware.
1414
- Next, we try to identify the parser from the provider, since these requests follow the pattern `/extensions/provider/webhook-path/`. If no parser is registered, we fall through this middleware
15-
- If we've found a parser ([`BaseRequestParser`](src/sentry/integrations/middleware/hybrid_cloud/parsers.py)), we defer to it for responding to the request, rather than falling through.
15+
- If we've found a parser ([`BaseRequestParser`](/src/sentry/integrations/middleware/hybrid_cloud/parser.py)), we defer to it for responding to the request, rather than falling through.
1616

1717
The parsers vary per integration but they follow the same basic steps:
1818

@@ -22,7 +22,7 @@ The parsers vary per integration but they follow the same basic steps:
2222
- Lastly, identify the relevant Cell Silos we need to forward to from looking at the `OrganizationMapping`s.
2323
- Now, depending on the payload we can choose how to respond to the initial request:
2424
- Some requests will require synchronous responses with an expected response pattern, (e.g. Slack).
25-
- Others don't care about the response, and we may opt to handle them asynchronously via the [`ControlOutbox` model](src/sentry/hybridcloud/models/outbox.py), (e.g. GitHub).
25+
- Others don't care about the response, and we may opt to handle them asynchronously via the [`ControlOutbox` model](/src/sentry/hybridcloud/models/outbox.py), (e.g. GitHub).
2626
- And others may require fanning out identical webhooks to multiple cells where the integration is installed on an organization, (e.g. Jira).
2727

2828
## Adding Integration Parsers
@@ -57,9 +57,12 @@ class ExampleRequestParser(BaseRequestParser):
5757

5858
```
5959

60-
You can register it and start forwarding requests when Sentry is running as a Control Silo by adding it to [`integeration_control.py`](src/sentry/middleware/integrations/integration_control.py):
60+
Put the parser in [`parsers/`](/src/sentry/middleware/integrations/parsers/), then register it by importing it and adding it to `active_parsers` in `IntegrationClassification.integration_parsers` in [`classifications.py`](/src/sentry/middleware/integrations/classifications.py):
6161

6262
```diff
63-
- ACTIVE_PARSERS = [SlackRequestParser]
64-
+ ACTIVE_PARSERS = [SlackRequestParser, ExampleRequestParser]
63+
active_parsers: list[type[BaseRequestParser]] = [
64+
BitbucketRequestParser,
65+
+ ExampleRequestParser,
66+
...
67+
]
6568
```

‎src/sentry/shared_integrations/client/integration_proxy_client.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ The proxying is managed by the [`IntegrationProxyClient`](/src/sentry/shared_int
1515
```
1616
{SENTRY_CONTROL_ADDRESS}/api/0/internal/integration-proxy # PROXY_BASE_PATH
1717
```
18-
The [integration proxy endpoint](/src/sentry/api/endpoints/internal/integration_proxy.py) parses the headers to verify the request is coming from a valid Region silo, and then replays the request with the proper authentication data (from `self.authorize_request`). The raw response is sent back to the originating silo to handle itself!
18+
The [integration proxy endpoint](/src/sentry/integrations/api/endpoints/integration_proxy.py) parses the headers to verify the request is coming from a valid Region silo, and then replays the request with the proper authentication data (from `self.authorize_request`). The raw response is sent back to the originating silo to handle itself!
1919

2020
## Implementing the IntegrationProxyClient
2121

‎src/sentry/workflow_engine/docs/conditions.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -520,8 +520,9 @@ condition handler alone cannot add detector packet data or detector-specific API
520520

521521
These are separate decisions, not part of backend registration.
522522

523-
For legacy issue-alert compatibility, add a forward translation in
524-
[`migration_helpers/issue_alert_conditions.py`](../migration_helpers/issue_alert_conditions.py).
523+
Production code no longer translates legacy issue-alert conditions. Test factories still do,
524+
through `data_condition_translator_mapping` in
525+
[`testutils/helpers/issue_alert_conditions.py`](../../testutils/helpers/issue_alert_conditions.py).
525526

526527
For the automation builder, add the matching frontend enum and node implementation in a
527528
separate frontend PR:
@@ -546,7 +547,7 @@ have explicit frontend nodes, defaults, details, and validation.
546547
| Add slow condition | `SLOW_CONDITIONS`, `event_frequency_query_handlers.py`, delayed query/result code, delayed processor tests |
547548
| Change API validation | `endpoints/validators/base/data_condition.py` and group/Detector/Workflow validators |
548549
| Change availability metadata | `endpoints/organization_data_condition_index.py` and handler serializer |
549-
| Add legacy compatibility | `migration_helpers/issue_alert_conditions.py` |
550+
| Add legacy test translation | `src/sentry/testutils/helpers/issue_alert_conditions.py` |
550551
| Add automation UI | Frontend condition enum, node registry, editor/details components, frontend tests |
551552

552553
## Conventions and Invariants

‎src/sentry/workflow_engine/docs/legacy_backport.md‎

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,8 @@ The deprecated legacy alert API routes were removed in
1212
the detector and workflow APIs to manage alerts. The serializer and issue-alert
1313
POST/PUT rollout flags no longer have consumers and have been retired.
1414

15-
Some compatibility code remains: rule history and statistics resolve legacy rule IDs
16-
to workflows, lookup endpoints expose associations between legacy and Workflow Engine
17-
models, and notification code still uses legacy-shaped data. Endpoint retirement does
15+
Some compatibility code remains: lookup endpoints expose associations between legacy and
16+
Workflow Engine models, and notification code still uses legacy-shaped data. Endpoint retirement does
1817
not imply that the association tables, serializers, or ID helpers are all unused.
1918

2019
## Handling IDs
@@ -41,8 +40,8 @@ Endpoints that accept IDs as input must handle both real legacy IDs (via associa
4140
## Compatibility Endpoints
4241

4342
Check the registered routes and their implementations when changing compatibility
44-
code. Rule history and statistics live in `src/sentry/rules/history/endpoints/`;
45-
association lookup routes live in `src/sentry/workflow_engine/endpoints/urls.py`.
43+
code. The association lookup routes (`alert-rule-workflow`, `alert-rule-detector`, and
44+
`incident-groupopenperiod`) live in `src/sentry/workflow_engine/endpoints/urls.py`.
4645
The endpoint-tracking decorator was removed with the deprecated routes.
4746

4847
## Unsupported legacy features

0 commit comments

Comments
 (0)