You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 9a8a89d
Browse filesBrowse the repository at this point in the historyBrowse files
docs(backend): Refresh backend agent and subsystem docs against code (#127884)
This is a docs-refresh pass over the backend developer docs. I checked
each claim in `AGENTS.md`, `src/AGENTS.md`, `tests/AGENTS.md`, and the
backend subsystem docs against the current code. Wrong claims are fixed,
and text that repeats something a lint, a runtime check, or another doc
already owns is cut. Frontend docs are out of scope.
Fixes that change what an agent or a human would actually do:
- **Tasks:** the guide described Celery (`queue`, `max_retries`). Tasks
run on taskbroker now, so it covers `namespace`, `retry=Retry(...)`, and
`TASKWORKER_IMPORTS` instead.
- **Silos:** `RegionSiloEndpoint` and `src/sentry/models/outbox.py`
don't exist. The guide now points to the
`@cell_silo_model`/`@control_silo_model` and endpoint decorators. It
also said org management lives in control, but `Organization` is a
cell-silo model.
- **Linting:** `prek run ruff` fails with "No hooks found". The hook id
is `ruff-check`.
- **Integrations:** there is no `integrations/registry.py`. Providers
register through `SENTRY_DEFAULT_INTEGRATIONS`.
- **Snuba CI:** the doc said Snuba CI runs `tests/snuba/`. It actually
picks tests by the `snuba_ci`/`snuba` markers.
- **Anti-patterns:** I removed "WRONG: direct model import in API".
About 135 of 158 files in `api/endpoints` do exactly that.
- **Backup tests:** the README described a
`JSONMutatingComparator`/`COMPARATORS` sentinel scheme that no longer
exists, and named the wrong snapshot file. It now describes
`JSONScrubbingComparator`, and it holds the add-a-model checklist that
used to sit in `tests/AGENTS.md`.
- **Subsystem docs:** I fixed moved paths for integration parsers and
the proxy endpoint, plus workflow engine references to the deleted
`migration_helpers/` and `rules/history/` modules.
I kept the existing layout (area `AGENTS.md` files, subsystem docs next
to code, skills for workflows) and did not add a top-level `docs/` tree.
Decisions for reviewers:
- The "import at top of module" and `hasattr`→`isinstance` rules could
be lint rules (`PLC0415`, or a custom check). If one gets enabled, its
prose line can be deleted.
- `.agents/skills/backend-conventions` mentions tracing helpers. I
didn't edit that skill here.
<!-- junior-request-attribution:start -->
via @joshuarli.
<!-- junior-request-attribution:end -->
<!-- junior-session-footer:start -->
<!-- junior-conversation-id:slack%3ACUHS29QJ0%3A1792184857.666969 -->
--
[View Junior
Session](https://junior-prod.sentry.dev/conversations/slack%3ACUHS29QJ0%3A1792184857.666969)
[[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3ACUHS29QJ0%3A1792184857.666969/?project=4510944073809921)
<!-- junior-session-footer:end -->
Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com>
Co-authored-by: Joshua Li <14209781+joshuarli@users.noreply.github.com>
- Never read `request.data["project_id"]`/`request.GET["project_id"]` directly. Use `self.get_projects(request=request, organization=organization, project_ids=...)`, which validates permissions.
13
+
- Never read `request.data["project_id"]`or`request.GET["project_id"]` directly. Use `self.get_projects(request=request, organization=organization, project_ids=...)`, which checks permissions.
14
14
15
15
For access-control review or deeper authorization work, use the **`django-access-review`** and **`sentry-security`** skills.
16
16
17
17
## Development Services
18
18
19
-
Local dependencies are managed by `devservices` (config: `devservices/config.yml`).
20
-
21
-
📖 Full devservices documentation: https://develop.sentry.dev/development-infrastructure/devservices.md
19
+
`devservices` runs local dependencies from `devservices/config.yml`. Full documentation: https://develop.sentry.dev/development-infrastructure/devservices/
22
20
23
21
## AI Assistant Quick Decision Trees
24
22
25
23
### "User wants to add an API endpoint"
26
24
27
-
1. Check if endpoint already exists: `grep -r "endpoint_name" src/sentry/api/`
2. Inherit from the scope base: `OrganizationEndpoint` (`src/sentry/api/bases/organization.py`) or `ProjectEndpoint` (`src/sentry/api/bases/project.py`).
27
+
3. Decorate the class with `@cell_silo_endpoint` or `@control_silo_endpoint` from `src/sentry/api/base.py`.
28
+
4. Put the endpoint in the owning domain's `endpoints/` package (for example `src/sentry/core/endpoints/`) or in `src/sentry/api/endpoints/`. Register the route in `src/sentry/api/urls.py`, or in the domain `urls.py` it includes. The test file mirrors the endpoint path under `tests/`.
29
+
5. Document it with drf-spectacular decorators → use the **`document-api-endpoint`** skill.
30
+
31
+
### "User wants to add a task"
32
+
33
+
Tasks run on taskbroker, not Celery.
34
+
35
+
1. Decorate the function with `@instrumented_task` from `src/sentry/tasks/base.py`. Pass `name`, a `namespace` from `src/sentry/taskworker/namespaces.py`, and `retry=Retry(...)` from `taskbroker_client.retry` when the task retries.
36
+
2. Add a new task module to `TASKWORKER_IMPORTS` in `src/sentry/conf/server.py`. `tests/sentry/taskworker/test_config.py` fails when a listed path does not import.
37
+
3. The test file mirrors the module path under `tests/`.
38
+
39
+
Full documentation: https://develop.sentry.dev/backend/application-domains/tasks/
45
40
46
41
### Serializers: Avoiding N+1 Queries
47
42
@@ -51,30 +46,25 @@ NEVER query the database in `serialize()` for bulk requests. `serialize()` runs
51
46
52
47
### API Documentation
53
48
54
-
- OpenAPI spec generation: `make build-api-docs`
55
-
- API ownership tracked in `src/sentry/apidocs/api_ownership_allowlist_dont_modify.py`
56
-
- Documenting/typing an endpoint → use the **`document-api-endpoint`** skill.
49
+
- Generate the OpenAPI spec with `make build-api-docs`.
50
+
- API ownership is tracked in `src/sentry/apidocs/api_ownership_allowlist_dont_modify.py`.
Viewer identity is wired through the app via the `ViewerContext` contextvar; use `sentry.viewer_context.get_viewer_context()` instead of explicitly threading org/user identity when the current viewer is in scope.
74
-
75
-
### Feature Flags
76
-
77
-
See the **feature-flags** skill (`.agents/skills/feature-flags/`) for registration, the `features.has(...)` check, and test usage.
67
+
The `ViewerContext` contextvar carries the current viewer's identity. When the viewer is in scope, call `sentry.viewer_context.get_viewer_context()` instead of passing org and user identity through arguments.
78
68
79
69
### Permissions
80
70
@@ -90,16 +80,15 @@ class MyPermission(SentryPermission):
90
80
91
81
### Logging, Tracing, Metrics, Options
92
82
93
-
Logging (`logger.info/exception`, `LOG005`/`LOG011`), tracing/spans (`sentry.utils.tracing`), metrics tag cardinality, and the options system (`options.get()`) → use the **`backend-conventions`** skill.
83
+
Logging (`LOG005`, `LOG011`), spans, metric tag cardinality, and `options.get()` → use the **`backend-conventions`** skill.
94
84
95
85
## Architecture Rules
96
86
97
87
### Silo Mode
98
88
99
-
-**Control Silo**: User auth, billing, organization management
100
-
-**Region Silo**: Project data, events, issues
101
-
- Check model's silo in `src/sentry/models/outbox.py`; use `@cell_silo_endpoint` or `@control_silo_endpoint`.
102
-
- Never join across silos; use `outbox` for cross-silo updates. For silo/RPC/outbox work, use the **`hybrid-cloud-rpc`**, **`hybrid-cloud-outboxes`**, and **`cell-architecture`** skills.
89
+
- The control silo holds global data such as `User`, `Integration`, and `OrganizationMapping`. The cell silo holds organization-scoped data such as `Organization`, `Project`, and `Group`.
90
+
- Each model declares its silo with `@control_silo_model` or `@cell_silo_model` (`src/sentry/db/models/base.py`). Each endpoint declares it with `@control_silo_endpoint` or `@cell_silo_endpoint`.
91
+
- Never join across silos. Use outboxes (`src/sentry/hybridcloud/models/outbox.py`) for cross-silo updates. For silo, RPC, and outbox work, use the **`hybrid-cloud-rpc`**, **`hybrid-cloud-outboxes`**, and **`cell-architecture`** skills.
**Every new Redis key sets a TTL, or is registered with Infrastructure Engineering as accepted durable data.**`CommonRedisCache.set` and `RedisKVStorage.set` raise `MissingTTL` rather than write a key with no expiry. There is no opt-out argument: the exemption is granted by Infrastructure Engineering, not at the callsite.
114
-
115
-
A bare `SET` or `GETSET` over an existing key clears its TTL; `SETEX` replaces it with the supplied expiry. `SADD`, `ZADD`, `HSET`, `HINCRBY` and `INCR` preserve an existing TTL. A TTL refreshed on every write does not bound the key's lifetime — shard by time window and give each shard a fixed TTL instead. Full rules: https://develop.sentry.dev/backend/application-domains/redis/.
102
+
**Every new Redis key sets a TTL, or is registered with Infrastructure Engineering as accepted durable data.**`CommonRedisCache.set` and `RedisKVStorage.set` raise `MissingTTL` for a key with no expiry, and there is no opt-out argument. Which commands drop an existing TTL, and why a sliding TTL is not a bound: https://develop.sentry.dev/backend/application-domains/redis/
116
103
117
104
## Anti-Patterns (NEVER DO)
118
105
119
106
```python
120
-
# WRONG: Direct model import in API
121
-
from sentry.models import Organization # NO!
122
-
123
-
# RIGHT: Use endpoint bases
124
-
from sentry.api.bases.organization import OrganizationEndpoint
125
-
126
-
# WRONG: Synchronous external calls
107
+
# WRONG: Slow external calls inside a request
127
108
response = requests.get(url) # NO!
128
109
129
-
# RIGHT: Use Celery task
130
-
from sentry.tasks import fetch_external_data
110
+
# RIGHT: Move the call into a task (see "User wants to add a task")
-**Feature Flags**: `src/sentry/features/permanent.py` or `temporary.py`
167
148
-**Utils**: `src/sentry/utils/{category}.py`
168
149
169
150
## Integration Development
170
151
171
-
Add an integration under `src/sentry/integrations/{name}/` with `__init__.py`, `integration.py` (inherit from `Integration`), `client.py`(API client), and `webhooks/` if needed. Register it in `src/sentry/integrations/registry.py` and add a feature flag in `temporary.py`.
152
+
Add an integration under `src/sentry/integrations/{name}/`. Subclass `IntegrationProvider` and `IntegrationInstallation`from `src/sentry/integrations/base.py`; `src/sentry/integrations/example/integration.py`is a working reference. Register the provider class path in `SENTRY_DEFAULT_INTEGRATIONS` in `src/sentry/conf/server.py`, and gate it with a flag in `src/sentry/features/temporary.py`.
Copy file name to clipboardExpand all lines: src/sentry/middleware/integrations/integration_control_middleware.md
+9-6Lines changed: 9 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,11 +8,11 @@ Incoming webhooks fired by integration providers notify us when changes occur in
8
8
9
9
## How it Works
10
10
11
-
The magic happens in the [`IntegrationControlMiddleware`](src/sentry/middleware/integrations/integration_control.py). Here, we do the following steps:
11
+
The magic happens in the [`IntegrationControlMiddleware`](/src/sentry/middleware/integrations/integration_control.py). Here, we do the following steps:
12
12
13
13
- If an HTTP request is received to `/extensions/*` (which is the prefix for all our webhooks) it is further inspected. If not, we fall through this middleware.
14
14
- Next, we try to identify the parser from the provider, since these requests follow the pattern `/extensions/provider/webhook-path/`. If no parser is registered, we fall through this middleware
15
-
- If we've found a parser ([`BaseRequestParser`](src/sentry/integrations/middleware/hybrid_cloud/parsers.py)), we defer to it for responding to the request, rather than falling through.
15
+
- If we've found a parser ([`BaseRequestParser`](/src/sentry/integrations/middleware/hybrid_cloud/parser.py)), we defer to it for responding to the request, rather than falling through.
16
16
17
17
The parsers vary per integration but they follow the same basic steps:
18
18
@@ -22,7 +22,7 @@ The parsers vary per integration but they follow the same basic steps:
22
22
- Lastly, identify the relevant Cell Silos we need to forward to from looking at the `OrganizationMapping`s.
23
23
- Now, depending on the payload we can choose how to respond to the initial request:
24
24
- Some requests will require synchronous responses with an expected response pattern, (e.g. Slack).
25
-
- Others don't care about the response, and we may opt to handle them asynchronously via the [`ControlOutbox` model](src/sentry/hybridcloud/models/outbox.py), (e.g. GitHub).
25
+
- Others don't care about the response, and we may opt to handle them asynchronously via the [`ControlOutbox` model](/src/sentry/hybridcloud/models/outbox.py), (e.g. GitHub).
26
26
- And others may require fanning out identical webhooks to multiple cells where the integration is installed on an organization, (e.g. Jira).
27
27
28
28
## Adding Integration Parsers
@@ -57,9 +57,12 @@ class ExampleRequestParser(BaseRequestParser):
57
57
58
58
```
59
59
60
-
You can register it and start forwarding requests when Sentry is running as a Control Silo by adding it to [`integeration_control.py`](src/sentry/middleware/integrations/integration_control.py):
60
+
Put the parser in [`parsers/`](/src/sentry/middleware/integrations/parsers/), then register it by importing it and adding it to `active_parsers` in `IntegrationClassification.integration_parsers` in [`classifications.py`](/src/sentry/middleware/integrations/classifications.py):
The [integration proxy endpoint](/src/sentry/api/endpoints/internal/integration_proxy.py) parses the headers to verify the request is coming from a valid Region silo, and then replays the request with the proper authentication data (from `self.authorize_request`). The raw response is sent back to the originating silo to handle itself!
18
+
The [integration proxy endpoint](/src/sentry/integrations/api/endpoints/integration_proxy.py) parses the headers to verify the request is coming from a valid Region silo, and then replays the request with the proper authentication data (from `self.authorize_request`). The raw response is sent back to the originating silo to handle itself!
0 commit comments