Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
166 lines (111 loc) · 10.3 KB

CONTRIBUTING.md

File metadata and controls

166 lines (111 loc) · 10.3 KB
Raw

Development

Here you'll find a contributing guide to get started with development.

Environment

For local development, you need a Python version that satisfies requires-python in pyproject.toml.

We use uv for project management. Install it and set up your IDE accordingly.

We use Poe the Poet as a task runner, similar to npm scripts in package.json. All tasks are defined in pyproject.toml under [tool.poe.tasks]. Run uv run poe to list them with their descriptions, and uv run poe <task> to run one.

Dependencies

To install this package with its development dependencies and the pre-commit hooks, run:

uv run poe install-dev

Code checking

To run all code checks (lint, type check, async docstring check, and unit tests), run:

uv run poe check-code

We use ruff for linting and formatting, ty for type checking, and pytest for tests. All of them are configured in pyproject.toml, and each has its own task (lint, format, type-check, unit-tests).

Tests

  • Unit tests (tests/unit) are isolated from the Apify API. See tests/unit/README.md for the rules they follow. Run them with uv run poe unit-tests, or a single test with uv run pytest tests/unit/<path>.py::<test_name>.
  • Integration tests (tests/integration) call the real Apify API. See tests/integration/README.md for the tokens they need. Run them with uv run poe integration-tests.

Code style

Readability

  • Prefer a few simple statements over one dense line. Give intermediate values a name instead of nesting calls, comprehensions, and unpacking into a single long expression.
  • When replacing a call with an equivalent one, change only the call and keep the surrounding statements as they were.

Comments and docstrings

  • Every module starts with from __future__ import annotations.
  • Comments and docstrings describe what the code does now. The bug a change fixed, what a PR changed, and contrasts with removed behavior (instead of ..., no longer ...) belong in the commit message.
  • Don't explain language features in comments. A comment earns its place with a domain reason, an invariant the types can't express, or a non-obvious consequence for callers.
  • Use single backticks for code references, never RST-style double backticks.
  • Wrap docstrings and prose comments to the full line length configured in pyproject.toml.
  • Write only ASCII in source files: -> instead of an arrow, - instead of a dash, straight quotes.
  • Write RQ for the request queue. Log messages and public .mdx docs spell out the full term.

Tests

  • Write flat def test_*() functions, never class Test*, and no section-header comments between tests.
  • Almost every test gets a one-line docstring stating what it verifies, without a Regression test:-style prefix.
  • In @pytest.mark.parametrize, wrap each case in pytest.param(..., id='...') with an id that describes the case.
  • Use Mock and the monkeypatch fixture for test doubles, not hand-rolled stubs, counters, or flags.
  • A test that changes process-wide or event-loop-wide state restores it. Prefer monkeypatch; for APIs it can't patch (loop.set_exception_handler(), signal.signal(), logging config, os.chdir()), capture the original value and restore it in a finally.
  • On Windows with Python 3.12 or older, datetime.now() ticks in 1-15 ms steps, so two timestamps taken a few milliseconds apart can be equal. A test that asserts one timestamp is later than another injects a Mock clock instead of sleeping.
  • For a TCP endpoint that must refuse connections, bind a socket to ('127.0.0.1', 0) and hold it open without calling listen(). A hard-coded port can be taken by a server in a parallel xdist worker. Test servers use 127.0.0.1, since on Windows localhost resolves to both IPv4 and IPv6.

YAML

Use double quotes for quoted YAML values (retries: "3"). Keep single quotes when the value itself contains double quotes, and for string literals inside GitHub Actions expressions (${{ ... }}, if: conditions), where double quotes are invalid.

Generated models

src/apify_client/_models.py, _typeddicts.py, and _literals.py are generated from the Apify OpenAPI specification by datamodel-code-generator and post-processed by scripts/postprocess_generated_models.py. Never edit them by hand; make lasting fixes in the post-processing script. The generator is configured under [tool.datamodel-codegen] in pyproject.toml, with field aliases in datamodel_codegen_aliases.json.

  • uv run poe generate-models regenerates them from the published specification. scripts/openapi_spec.py describes how the specification version is recorded. Don't commit a regeneration that only moves that version.
  • uv run poe generate-models-from-file <path> generates them from a local specification, for trying out unpublished changes. Don't commit the result.
  • The nightly Regenerate models workflow opens a chore: PR when the models change. If the diff is user-facing, retitle it to fix: or feat: so it lands in the changelog and triggers a release.

Documentation

We adhere to the Google docstring format for documenting our codebase. Every user-facing class or method is documented. Documentation standards are enforced using Ruff.

Resource client docstrings follow one pattern:

  • The first line is Sub-client for managing a specific <resource>. or Sub-client for the <resource> collection.
  • The second paragraph names two or three things the client does (not every method) and says to obtain an instance via an appropriate method on the parent client class, without naming that method.

Docstrings are written on the sync clients only. After editing one, run uv run poe fix-docstrings to copy it to the async client; uv run poe check-docstrings fails when the two differ.

Our API documentation is generated from these docstrings using pydoc-markdown with additional post-processing. Markdown files in the docs/ folder complement the autogenerated content. Final documentation is rendered using Docusaurus and published to GitHub Pages.

A user-facing feature ships with its documentation in the same PR: a page in docs/02_concepts/ that explains the mechanism and, where it helps, a page in docs/03_guides/ that shows a realistic use.

Code examples in docs/ come in _sync.py / _async.py pairs and have their own lint settings in docs/pyproject.toml. Never edit website/versioned_docs/: those snapshots are produced by the release automation and frozen against the code that shipped. Fix docs/, and the next release snapshot carries the change.

To run the documentation locally (requires Node.js):

uv run poe run-docs

Linting the docs and website

Markdown content (this guide, README.md, and the docs/ folder) is checked with markdownlint. The Docusaurus website code is linted with oxlint in type-aware mode (types come from website/tsconfig.json) and formatted with oxfmt. All of them run in CI. To run them locally (requires Node.js 22.12 or newer and pnpm), from the website/ directory:

pnpm lint          # lint Markdown and website code
pnpm lint:fix      # auto-fix both
pnpm format        # format the website code

Doc images are committed as optimized .webp. To convert a new image, run pnpm opt:images <path-to-image> from the website/ directory.

Commits

We use Conventional Commits format for commit messages. The release process reads them to determine the version bump and build the changelog. Pick the type by what the commit changes, not by why:

Type Description
feat A new feature in the source code
fix A bug fix in the source code
perf A source code change that improves performance
refactor A source code change that neither fixes a bug nor adds a feature
style A source code change that doesn't affect its meaning (white-space, formatting)
test Adding or changing tests
docs Documentation changes
ci CI workflow changes
build Build system changes
chore Dependency bumps, tooling, generated files, and other housekeeping
revert Reverting a previous commit

feat, fix, perf, refactor, and style trigger a release and appear in the changelog, so use them for source code changes only. docs triggers a docs release on master.

Logging changes are breaking: changing a log level, a logger name, or the wording of a log message, or removing one, breaks downstream log-based tooling. Keep the accurate type, add ! after it, and list the changes in a BREAKING CHANGE: footer. PRs are squash-merged, so the ! must also be in the PR title.

Release process

Publishing new versions to PyPI is automated through GitHub Actions.

  • Beta releases: On each commit to the master branch, a new beta release is automatically published. The version number is determined based on the latest release and conventional commits. The beta version suffix is incremented by 1 from the last beta release on PyPI.
  • Stable releases: A stable version release may be created by triggering the release GitHub Actions workflow. The version number is determined based on the latest release and conventional commits (auto release type), or it may be overridden using the custom release type.

Publishing to PyPI manually

  1. Do not do this unless absolutely necessary. In all conceivable scenarios, you should use the release workflow instead.

  2. Make sure you know what you're doing.

  3. Update the version number by modifying the version field under project in pyproject.toml:

    [project]
name = "apify_client"
version = "x.z.y"

  4. Build the package:

    uv run poe build

  5. Upload to PyPI:

    uv publish --token YOUR_API_TOKEN