Skip to content

Commit ed8c25d

Browse files
authored
docs: consolidate contributing guide and agent rules (#1124)
Makes `CONTRIBUTING.md` the single source of development conventions. `.rules.md` (the target of `CLAUDE.md`, `AGENTS.md` and `GEMINI.md`) links and imports it and keeps only the architecture notes. - Removes duplication between `.rules.md`, `CONTRIBUTING.md`, `README.md` and `pyproject.toml`. Poe tasks have `help` texts, so `uv run poe` replaces the task table. - Adds code style, test, docs and commit conventions agreed in past reviews, plus a section on generated models. - Fixes drift in the architecture notes. *✍️ Drafted by Claude Code*
1 parent 972c549 commit ed8c25d

6 files changed

Lines changed: 110 additions & 199 deletions

File tree

‎.rules.md‎

Lines changed: 18 additions & 97 deletions
Original file line numberDiff line numberDiff line change
@@ -2,122 +2,43 @@
22

33
This file provides guidance to programming agents when working with code in this repository.
44

5-
## Project Overview
5+
Python client for the [Apify API](https://docs.apify.com/api/v2), published on PyPI as `apify-client`. `ApifyClient` (sync) and `ApifyClientAsync` (async) are the entry points.
66

7-
Python client for the [Apify API](https://docs.apify.com/api/v2). Provides `ApifyClient` (sync) and `ApifyClientAsync` (async) classes to interact with the Apify platform. Published on PyPI as `apify-client`.
7+
Read [CONTRIBUTING.md](CONTRIBUTING.md) first. It covers setup, tasks, tests, code style, generated models, documentation, and commits, and all of it applies to agents too.
88

9-
## Development Commands
10-
11-
Uses [uv](https://docs.astral.sh/uv/) for project management and [Poe the Poet](https://poethepoet.natn.io/) as task runner. Requires Python 3.11+.
12-
13-
```bash
14-
uv run poe install-dev # Install dev deps + git hooks
15-
uv run poe check-code # Run all checks (lint, type-check, unit-tests, docstring check)
16-
uv run poe lint # Ruff format check + ruff check
17-
uv run poe format # Auto-fix lint issues and format
18-
uv run poe type-check # Run ty type checker
19-
uv run poe unit-tests # Run unit tests
20-
uv run poe check-docstrings # Verify async docstrings match sync
21-
uv run poe fix-docstrings # Auto-fix async docstrings
22-
uv run poe generate-models # Regenerate _models.py, _typeddicts.py, _literals.py from the published spec
23-
uv run poe generate-models-from-file <path> # Regenerate from a local OpenAPI spec file
24-
25-
# Run a single test
26-
uv run pytest tests/unit/test_file.py
27-
uv run pytest tests/unit/test_file.py::test_name
28-
```
29-
30-
Integration tests require `APIFY_TEST_USER_API_TOKEN` and `APIFY_TEST_USER_2_API_TOKEN` env vars.
9+
@CONTRIBUTING.md
3110

3211
## Architecture
3312

34-
### Client Hierarchy
13+
### Client hierarchy
3514

36-
`ApifyClient`/`ApifyClientAsync` are the entry points. They provide methods that return resource sub-clients:
15+
Entry points return resource sub-clients, which can create child clients (e.g. `ActorClient.builds()` -> `BuildCollectionClient`):
3716

3817
```text
3918
ApifyClient
40-
├── .actor(id) → ActorClient (single resource operations)
19+
├── .actor(id) → ActorClient (single resource operations)
4120
├── .actors() → ActorCollectionClient (list/create)
42-
├── .dataset(id) → DatasetClient
43-
├── .datasets() → DatasetCollectionClient
4421
├── .run(id) → RunClient
45-
├── .runs() → RunCollectionClient
46-
└── ... (schedules, tasks, webhooks, key-value stores, request queues, etc.)
22+
└── ... (datasets, schedules, tasks, webhooks, key-value stores, request queues, etc.)
4723
```
4824

49-
Each resource client can create child clients (e.g., `ActorClient.builds()` → `BuildCollectionClient`).
50-
51-
### Sync/Async Symmetry
52-
53-
Every resource client has both sync and async variants in the **same file**. For example, `actor.py` contains both `ActorClient` and `ActorClientAsync`. Both share the same interface — async versions use `async/await`.
54-
55-
Docstrings are written on sync clients and **automatically copied** to async clients via `scripts/check_docstrings.py`. When modifying docstrings, edit only the sync client and run `uv run poe fix-docstrings`.
25+
`ClientRegistry` / `ClientRegistryAsync` (`_client_registry.py`) hold the resource client classes and are passed to every client, so clients can create child clients without importing them (avoids circular imports). All resource clients derive from `ResourceClientBase` (`_resource_clients/_resource_client.py`), whose `WithLogDetailsClient` metaclass wraps public methods for structured logging.
5626

57-
### Dependency Injection via ClientRegistry
27+
### Sync/async symmetry
5828

59-
`ClientRegistry`/`ClientRegistryAsync` (in `_client_registry.py`) holds references to all resource client classes and is passed to each client. This avoids circular imports and lets resource clients create child clients without directly importing them.
29+
Every resource client in `src/apify_client/_resource_clients/` has its sync and async variant in the **same file** (e.g. `ActorClient` and `ActorClientAsync` in `actor.py`) with the same interface. Docstrings are written on the sync client only (see [Documentation](CONTRIBUTING.md#documentation)).
6030

61-
### HTTP Client Abstraction
31+
### HTTP layer
6232

63-
- `HttpClient`/`HttpClientAsync` — base classes in `http_clients/_base.py` holding the shared request pipeline (retries,
64-
timeouts, API errors); transports implement the `send_request`, error-classification, and lifecycle hooks
65-
- `ImpitHttpClient`/`ImpitHttpClientAsync` — default implementation (Rust-based Impit)
66-
- `Httpx2HttpClient`/`Httpx2HttpClientAsync` — built-in alternative behind the `httpx2` optional extra
67-
- `HttpResponse` — Protocol (not a concrete class) for response objects
68-
- Users can plug in custom HTTP clients via `ApifyClient.with_custom_http_client()`
33+
`HttpClient` / `HttpClientAsync` (`http_clients/_base.py`) hold the shared request pipeline (retries, timeouts, API errors); transports (`Impit*` default, `Httpx2*` behind the `httpx2` extra, or a custom one via `ApifyClient.with_custom_http_client()`) implement sending, error classification, and lifecycle hooks. Request-body compression lives in `http_compressors/`, selected by the `compression` constructor argument.
6934

70-
### Base Classes
35+
### Generated models
7136

72-
- `ResourceClientBase` — shared URL building, parameter handling, metaclass-based logging
73-
- `ResourceClient` / `ResourceClientAsync` — sync/async base classes with HTTP call methods
74-
- `WithLogDetailsClient` metaclass — auto-wraps public methods for structured logging
37+
`_models.py`, `_typeddicts.py`, and `_literals.py` are **generated**; never edit them by hand (see [Generated models](CONTRIBUTING.md#generated-models)). Each input-side TypedDict ships in a snake_case and a camelCase variant (`RequestResourceDict`, `RequestResourceCamelDict`); the camel variants are synthesized in the post-processing script, and resource-client signatures accept either casing or the Pydantic model.
7538

76-
### Data Models
39+
### Checking API response shapes
7740

78-
`src/apify_client/_models.py` and `src/apify_client/_typeddicts.py` are **auto-generated** — do not edit them manually. Every Pydantic model and TypedDict comes from the OpenAPI spec.
41+
The OpenAPI spec can lag behind the API and omit fields, and a live response only shows the fields that happen to be set. If the private `apify-core` repository is available locally, use it to confirm which fields an endpoint returns:
7942

80-
Each input-side TypedDict ships in two casings: snake_case (`RequestResourceDict`) and camelCase (`RequestResourceCamelDict`). The camel variants are synthesized in `scripts/postprocess_generated_models.py` from the Pydantic `Field(alias=...)` map; resource-client signatures union both with the Pydantic model so users can pass either casing without losing type-checker support.
81-
82-
- Generated by `datamodel-code-generator` from the OpenAPI spec at `https://docs.apify.com/api/openapi.json` (config in `pyproject.toml` under `[tool.datamodel-codegen]`, aliases in `datamodel_codegen_aliases.json`)
83-
- After generation, `scripts/postprocess_generated_models.py` is run to apply additional fixes
84-
- To regenerate locally:
85-
- From the live published spec: `uv run poe generate-models`
86-
- From a local spec file: `uv run poe generate-models-from-file path/to/openapi.json`
87-
- `scripts/openapi_spec.py` downloads the spec once into git-ignored `tmp/openapi.json` — both codegen passes read that copy, so a spec redeployed mid-run can't leave the generated files describing two different inputs — then records its `info.version` in `pyproject.toml` under `[tool.apify.openapi-spec]`
88-
- That version is a **coarse marker, not a content identity**: the spec is served latest-only, and apify-docs bumps `components/version.yaml` in a follow-up `[skip ci]` commit, so a deploy can publish new content under the old stamp. A moved value proves the spec changed; an unchanged value proves nothing
89-
- Recording is unconditional; the nightly workflow keeps it honest by committing nothing unless the models changed. A local `generate-models` that only moves that line should be dropped, not committed alone. `generate-models-from-file` leaves the line alone, so models from a candidate spec must not be committed
90-
- In CI, the `Regenerate models` workflow (`on_schedule_regenerate_models.yaml`) runs nightly at 02:00 UTC and opens a PR when the models change. It always generates on master and rebuilds its `ci/regenerate-models` branch from master instead of appending, so the PR diff is always "current spec vs current master". Retitle the `chore:` PR to `fix:`/`feat:` when the diff is user-facing, so it lands in the changelog and triggers a release
91-
- The gate is the **generated models**, not the spec version — which moves on rebuilds regardless of client impact, and doesn't reliably move when the content does
92-
- Manual regeneration is also possible from the GitHub Actions UI (`Regenerate models` workflow)
93-
94-
## Code Conventions
95-
96-
- **Line length**: 120 characters
97-
- **Linting/formatting**: Ruff with nearly all rules enabled (see `pyproject.toml`)
98-
- **Type checking**: ty (Astral's type checker)
99-
- **Docstrings**: Google style format, single backticks for inline code references (`` `ApifyClient` `` not ``` ``ApifyClient`` ```)
100-
- **Imports**: `from __future__ import annotations` used throughout
101-
- **Commits**: [Conventional Commits](https://www.conventionalcommits.org/) format. Choose the type based on *what* changed, not just *why*:
102-
- `feat:` / `fix:` / `perf:` / `refactor:` / `style:` — **source code only**; these trigger a release and appear in the changelog
103-
- `test:` — test additions or changes (no release triggered)
104-
- `docs:` — documentation changes; also triggers a doc release on master
105-
- `ci:` — CI/workflow changes
106-
- `chore:` — dependency bumps, tooling, generated files (e.g. model regeneration PRs), and other housekeeping
107-
- `build:` — build system changes
108-
109-
## Testing
110-
111-
- **Unit tests** (`tests/unit/`): Use `pytest-httpserver` to mock HTTP. No network required.
112-
- **Integration tests** (`tests/integration/`): Hit the live Apify API.
113-
- **Async**: `asyncio_mode = "auto"` — async tests run automatically without markers.
114-
- **Parallelism**: Tests run in parallel via `pytest-xdist`.
115-
116-
Unit test pattern:
117-
118-
```python
119-
def test_example(httpserver: HTTPServer) -> None:
120-
httpserver.expect_request('/v2/endpoint').respond_with_json({'data': ...})
121-
client = ApifyClient(token='test', api_url=httpserver.url_for('/').removesuffix('/'))
122-
# assert client behavior
123-
```
43+
- A single resource's fields are in `PUBLISHED_*_FIELDS` in `src/packages/<area>-server/src/<resource>/<resource>.server.ts`.
44+
- A list endpoint has its own, usually smaller, `API_PUBLISHED_*_LIST_FIELDS` in `src/api/src/routes/<resource>/<resource>_list.ts`, and often builds each item inline. Don't assume a list item has the same fields as the single resource.

0 commit comments

Comments
 (0)