diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b6638ea4..f27f54fe 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -30,8 +30,10 @@ We use [ruff](https://docs.astral.sh/ruff/) for linting and formatting, [ty](htt ## Tests -- **Unit tests** (`tests/unit`) are isolated from the Apify API. See [`tests/unit/README.md`](tests/unit/README.md) for the rules they follow. Run them with `uv run poe unit-tests`, or a single test with `uv run pytest tests/unit/.py::`. -- **Integration tests** (`tests/integration`) call the real Apify API. See [`tests/integration/README.md`](tests/integration/README.md) for the tokens they need. Run them with `uv run poe integration-tests`. +[`tests/README.md`](tests/README.md) describes each test level, with its rules and required tokens. + +- **Unit tests** (`tests/unit`) are isolated from the Apify API. Run them with `uv run poe unit-tests`, or a single test with `uv run pytest tests/unit/.py::`. +- **Integration tests** (`tests/integration`) call the real Apify API. Run them with `uv run poe integration-tests`. ## Code style diff --git a/tests/README.md b/tests/README.md new file mode 100644 index 00000000..760b8184 --- /dev/null +++ b/tests/README.md @@ -0,0 +1,87 @@ +# Tests + +The tests have two levels: + +- [Unit tests](#unit-tests) (`tests/unit`) are isolated from the Apify API and need no credentials or network access. +- [Integration tests](#integration-tests) (`tests/integration`) call the real Apify API. + +A test that calls the Apify API is an integration test. Every other test is a unit test. Helpers shared by both levels live in `tests/_utils.py`. + +## Unit tests + +Unit tests are fully isolated from the real Apify API. They use mocks, fakes, or a local HTTP server, so they need no API tokens and no network access. + +### Running unit tests + +```bash +uv run poe unit-tests +``` + +To run a single test: + +```bash +uv run pytest tests/unit/.py:: +``` + +### Unit test fixtures + +- `httpserver`: a local HTTP server (from `pytest-httpserver`) on `127.0.0.1`, shared by the session and cleared after each test. +- `sync_client` and `async_client`: an `ApifyClient` and an `ApifyClientAsync` pointed at `httpserver`. +- `http_client_classes`: runs the test once per built-in HTTP transport (`impit` and `httpx2`) and makes it the default transport the clients construct. A module opts all its tests in with `pytestmark = pytest.mark.usefixtures('http_client_classes')`. Tests that construct a transport directly use `http_client_class` and `http_client_async_class`. + +### Mocking the API + +Register the expected requests on `httpserver` and point the client at it: + +```python +def test_example(httpserver: HTTPServer, sync_client: ApifyClient) -> None: + httpserver.expect_request('/v2/endpoint').respond_with_json({'data': ...}) + # call sync_client and assert its behavior +``` + +## Integration tests + +Integration tests communicate with the real Apify API. + +### Running integration tests + +The tests need API tokens of two test users: + +- `APIFY_TEST_USER_API_TOKEN`: the main test user. +- `APIFY_TEST_USER_2_API_TOKEN`: a second user, who owns the storages for the permission tests. + +```bash +export APIFY_TEST_USER_API_TOKEN= +export APIFY_TEST_USER_2_API_TOKEN= +uv run poe integration-tests +``` + +To run them against a different Apify API environment, also set `APIFY_INTEGRATION_TESTS_API_URL`. + +### Integration test fixtures + +- `client`: an `ApifyClient` or `ApifyClientAsync` for the main test user. Each test that uses it runs four times, once for each combination of sync or async client and `impit` or `httpx2` transport. +- `is_async`: `True` when the test runs with the async client. +- `test_dataset_of_another_user` and `test_kvs_of_another_user`: a dataset and a key-value store owned by the second user, with known content and the signatures for accessing it. + +### Writing integration tests + +One test function covers both clients. Wrap each client call in `maybe_await`, which awaits the result of the async client and returns the result of the sync one as is: + +```python +async def test_dataset_get_or_create(client: ApifyClient | ApifyClientAsync) -> None: + """A dataset created by name can be retrieved by its ID.""" + dataset_name = get_random_resource_name('dataset') + dataset = await maybe_await(client.datasets().get_or_create(name=dataset_name)) + dataset_client = client.dataset(dataset.id) + + try: + retrieved = await maybe_await(dataset_client.get()) + assert retrieved is not None + assert retrieved.name == dataset_name + finally: + await maybe_await(dataset_client.delete()) +``` + +- Name the resources a test creates with `get_random_resource_name`, so parallel test runs don't collide, and delete them in a `finally` block. +- The API is eventually consistent, so a resource you just created or changed may take a moment to show up. Poll with `poll_until_condition`, or with `collect_iterate_until_present` for `iterate()` listings, instead of sleeping for a fixed time. Where a test has to sleep, use `maybe_sleep`, which blocks only for the sync client. diff --git a/tests/integration/README.md b/tests/integration/README.md deleted file mode 100644 index afde0f52..00000000 --- a/tests/integration/README.md +++ /dev/null @@ -1,11 +0,0 @@ -# Integration tests - -Integration tests are expected to communicate with the real Apify API. -They require valid API tokens provided via environment variables: - -- `APIFY_TEST_USER_API_TOKEN` -- `APIFY_TEST_USER_2_API_TOKEN` (used for permission-related tests) - -To run them against a different Apify API environment, also set `APIFY_INTEGRATION_TESTS_API_URL`. - -Tests that do not call the Apify API should be implemented as unit tests instead. diff --git a/tests/unit/README.md b/tests/unit/README.md deleted file mode 100644 index f2c7ea02..00000000 --- a/tests/unit/README.md +++ /dev/null @@ -1,20 +0,0 @@ -# Unit tests - -Unit tests must be fully isolated from the real Apify API. -Use mocks, fakes, or a local HTTP test server. - -They must not require: - -- production or test API tokens -- network access at all - -Any test that calls the Apify API belongs to integration tests. - -Mock HTTP with the `httpserver` fixture from `pytest-httpserver`: - -```python -def test_example(httpserver: HTTPServer) -> None: - httpserver.expect_request('/v2/endpoint').respond_with_json({'data': ...}) - client = ApifyClient(token='test', api_url=httpserver.url_for('/').removesuffix('/')) - # assert client behavior -```