You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit ff26ba9
Browse filesBrowse the repository at this point in the historyBrowse files
docs: [Sample Tests 40] Describe the in-module end-to-end tests instead of the system tests (#6332)
* docs: Describe the in-module end-to-end tests instead of the system tests
The e2e_tests rule now covers MockSentryServer, where each kind of
end-to-end test lives, the Spring suites per flavor, the OpenTelemetry
agent fixture, how to run them and the gotchas the migration found. The
test skill runs integration suites by their source set instead of the
Python runner, the new_module rule no longer asks for sample tests, and
AGENTS.md drops sentry-system-test-support.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Merge the audit fixes from the branches below, and document the test client and log4j2 suite
The context's TestRestTemplate adds trace headers of its own whenever
Sentry customizes the RestTemplateBuilder, not only in OpenTelemetry
mode, and the Spring Boot modules have an integrationTestLog4j2 suite.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: .claude/skills/test/SKILL.md
+16-22Lines changed: 16 additions & 22 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,13 +1,14 @@
1
1
---
2
2
name: test
3
-
description: Run tests for a specific SDK module. Use when asked to "run tests", "test module", "run unit tests", "run system tests", "run e2e tests", or test a specific class. Auto-detects unit vs system tests. Supports interactive mode.
3
+
description: Run tests for a specific SDK module. Use when asked to "run tests", "test module", "run unit tests", "run integration tests", "run e2e tests", or test a specific class. Auto-detects unit tests vs integration test suites. Supports interactive mode.
Run tests for a specific module. Auto-detects whether to run unit tests or system tests.
10
+
Run tests for a specific module. Auto-detects whether to run unit tests or an integration test
11
+
suite.
11
12
12
13
## Step 0: Check for Interactive Mode
13
14
@@ -19,7 +20,7 @@ In interactive mode, use AskUserQuestion at decision points as described in the
19
20
20
21
The argument can be either:
21
22
- A **file path** (e.g., `@sentry/src/test/java/io/sentry/ScopesTest.kt`)
22
-
- A **module name** (e.g., `sentry-android-core`, `sentry-samples-spring-boot-4`)
23
+
- A **module name** (e.g., `sentry-android-core`, `sentry-spring-boot-4`)
23
24
- A **module name + test filter** (e.g., `sentry ScopesTest`)
24
25
25
26
Extract the module name and optional test class filter from the argument.
@@ -30,10 +31,13 @@ Extract the module name and optional test class filter from the argument.
30
31
31
32
| Signal | Test Type |
32
33
|--------|-----------|
33
-
| Path contains `sentry-samples/`| System test |
34
-
|Module name starts with `sentry-samples-`|System test|
34
+
| Path contains `/src/integrationTest*/` or `/src/agentFixture/`| Integration test suite|
35
+
|Path contains `sentry-samples/`, or module name starts with `sentry-samples-`|None: samples have no tests|
35
36
| Everything else | Unit test |
36
37
38
+
For a sample, tell the user that samples have no tests, and that the end-to-end tests for what
39
+
the sample shows live in the module it uses (see the `e2e_tests` rule).
40
+
37
41
## Step 3a: Run Unit Tests
38
42
39
43
Determine the Gradle test task:
@@ -60,30 +64,20 @@ Without a filter:
60
64
./gradlew ':<module>:<task>' --info
61
65
```
62
66
63
-
## Step 3b: Run System Tests
64
-
65
-
System tests require the Python-based test runner which manages a mock Sentry server and sample app lifecycle.
66
-
67
-
1. Ensure the Python venv exists:
68
-
```bash
69
-
test -d .venv || make setupPython
70
-
```
67
+
## Step 3b: Run an Integration Test Suite
71
68
72
-
2. Extract the sample module name. For file paths like `sentry-samples/<sample-module>/src/...`, the sample module is the directory name (e.g., `sentry-samples-spring`).
69
+
The suite's task is named after its source set directory: `src/integrationTestWebflux/...` runs
70
+
with `integrationTestWebflux`. The OpenTelemetry agent suite runs in two tasks, one per init mode,
71
+
so for `src/integrationTestOtelAgent/` or `src/agentFixture/` run both:
73
72
74
-
3. Run the system test:
75
73
```bash
76
-
.venv/bin/python test/system-test-runner.py test --module<sample-module>
This starts the mock Sentry server, starts the sample app (Spring Boot/Tomcat/CLI), runs tests via `./gradlew :sentry-samples:<sample-module>:systemTest`, and cleans up afterwards.
80
-
81
-
To run **every** system test instead of one module, use the Makefile targets — they also create the
82
-
venv for you:
77
+
Otherwise, with or without a test class filter:
83
78
84
79
```bash
85
-
make systemTest # all system tests (--all)
86
-
make systemTestInteractive # pick the setups to run (--interactive)
Copy file name to clipboardExpand all lines: .cursor/rules/e2e_tests.mdc
+70-14Lines changed: 70 additions & 14 deletions
Original file line number
Diff line number
Diff line change
@@ -3,26 +3,82 @@ alwaysApply: false
3
3
description: Java SDK End to End Tests
4
4
---
5
5
6
-
# Java SDK End to End Tests (System Tests)
6
+
# Java SDK End to End Tests
7
7
8
-
The samples in the `sentry-samples` directory are used to run end to end tests against them.
8
+
End-to-end tests live in the module whose code they test, not in `sentry-samples`. Samples have no
9
+
tests. A test runs the code the way an app would and asserts what reaches Sentry over the real HTTP
10
+
transport.
9
11
10
-
There is a python script (`system-test-runner.py`) that can be used to run one (using `--module SAMPLE_NAME`) or all (using `--all`) system tests.
12
+
## MockSentryServer
11
13
12
-
The script has an interactive mode (`-i`) which allows selection of test setups to execute, whether to run the tests or just prepare infrastructure for testing from IDE.
14
+
`MockSentryServer` (`sentry-test-support`) is an in-process HTTP server on a free port. Point the
15
+
SDK at `server.dsn`, then:
13
16
14
-
The tests run a mock Sentry server via `system-test-sentry-server.py`. Any system under test will then have a DSN set that reflects this local mock server like `http://502f25099c204a2fbf4cb16edc5975d1@localhost:8000/0`.
15
-
By using this local DSN, the system under test sends events to the local mock server.
16
-
The tests can then use `TestHelper` to assert envelopes that were received by the mock server.
17
-
`TestHelper` uses HTTP requests to retrieve the JSON payload of the received events and deserialize them back to objects for easier assertion.
18
-
Tests can then assert events, transactions, logs etc. similar to how they would appear in `beforeSend` and similar callbacks.
17
+
- `awaitEnvelope(timeoutMillis) { predicate }` polls for a matching envelope and fails with what it
18
+
received on timeout.
19
+
- `assertNoEnvelope(millis) { predicate }` checks that nothing matching arrives.
20
+
- `MockSentryServer { ... }` takes a hook that runs before each check while waiting, for example to
21
+
flush an OpenTelemetry tracer provider.
19
22
20
-
`TestHelper` has a lot of helper methods for asserting, e.g. by span name, log body etc.
23
+
`EnvelopeMatchers.kt` has the accessors and matchers: `event()`, `transaction()`, `logs()`,
24
+
`metrics()`, `profileChunk()`, `hasSpanWithOp`, `hasFlag`, `hasLogWithAttribute`, `hasMetric` and
25
+
so on.
21
26
22
-
The end to end tests either expect the system under test to either be running on a server or call `java -jar` to execute a CLI system under test.
27
+
Call `Sentry.flush()` before awaiting when Sentry runs in the test's JVM: logs and metrics are
28
+
batched. Envelopes that arrive late can outlive a server's `reset()`, so tests that share a server
29
+
match on values only they produce.
23
30
24
-
For Spring Boot, we spin up the Spring Boot server. The tests then send requests to that server and assert what is sent to Sentry.
31
+
## Where the tests are
25
32
26
-
End to end tests are also executed on CI using a matrix build, as defined in `.github/workflows/system-tests-backend.yml`.
33
+
- **Core and plain integrations** use their module's `src/test`. Examples: `SentryEndToEndTest`
34
+
(`sentry`), `SentryAppenderIntegrationTest` (`sentry-logback`, `sentry-log4j2`, loading the
35
+
appender from XML), `SentryHandlerIntegrationTest` (`sentry-jul`),
-**`sentry-test-support`** - Shared test utilities
113
-
-**`sentry-system-test-support`** - System testing infrastructure
112
+
-**`sentry-test-support`** - Shared test utilities, including `MockSentryServer` and envelope matchers
114
113
-**`sentry-samples`** - Example applications
115
114
-**`sentry-bom`** - Bill of Materials for dependency management
116
115
@@ -158,7 +157,7 @@ PR description why the broad catch is necessary.
158
157
### Testing Requirements
159
158
- Write comprehensive unit tests for new features
160
159
- Android modules require both unit tests and instrumented tests where applicable
161
-
-System tests validate end-to-end functionality with sample applications
160
+
-End-to-end tests run in the module they test, against `MockSentryServer`. Samples have no tests
162
161
-**Assertions**: For new unit tests, prefer [Google Truth](https://truth.dev/) (`com.google.common.truth.Truth.assertThat`) over `kotlin.test`/JUnit assertions for its readable, fluent API. Keep using `kotlin.test` for test structure (`@Test`, `assertFailsWith`). See `sentry/src/test/java/io/sentry/DsnTest.kt` for the style. Don't rewrite existing `kotlin.test` assertions solely to switch libraries.
163
162
- Truth is wired into the `sentry` module. When adding Truth-based tests to another module, add `testImplementation(libs.google.truth)` to that module's `build.gradle.kts`.
0 commit comments