chore: add agents.md & add linting rules TLDR (#33)

Author: mrapplexz

AGENTS.md

@@ -0,0 +1,57 @@
+# AGENTS.md
+
+Guidance for AI agents working in the **d9d** repository.
+
+## Read these first
+
+Most conventions are already documented:
+
+- `README.md` - project purpose, philosophy, and what d9d is/isn't.
+- `CONTRIBUTING.md` - the canonical reference. It covers:
+  - Development setup.
+  - The `Makefile` workflow.
+  - Design Principles. Treat these as hard rules when writing or changing code.
+  - Linting (`ruff`), type checking (`ty`), testing tiers, and docstring style.
+  - The DEP process for major changes.
+  - Conventional Commits format and the PR checklist.
+- `deps/0001-dep-process.md` - when and how to write a D9D Enhancement Proposal.
+- `pyproject.toml` - authoritative source for enabled `ruff` rules, `ty` config, and dependencies.
+- `docs/` - user-facing documentation, mirrors the package layout in `d9d/`.
+  - `docs/index.md` - user-facing intro.
+  - `docs/toc.md` - annotated map of every subsystem and its docs page.
+
+## Where things live
+
+- `d9d/` - library source. Mirror its layout when adding docs in `docs/`.
+- `test/d9d_test/` - tests. `-m local` (single process) and `-m distributed` (require `torchrun`).
+- `deps/` - enhancement proposals.
+- `example/` - runnable training examples.
+- `packages/` - manually-built optional dependencies (see `compat-local-overrides` in [CONTRIBUTING.md](./CONTRIBUTING.md)).
+
+### Source layout (`d9d/`)
+
+Top-level packages of the library.
+
+- `core/` - distributed primitives: `dist_context` (the `DeviceMesh` source of truth), `dist_ops`, `sharding` (PyTree sharding), `offload` (sleep/wake state offloading), `autograd`, `protocol`, `types`.
+- `loop/` - execution engine: the `Trainer`/`Inference` lifecycle, dependency injection, config schemas, and run/control/event machinery (`auto`, `component`, `config`, `control`, `event`, `run`).
+- `module/` - modeling building blocks: `base`, `block`, `model` (model catalogue), and `parallelism`.
+- `pipelining/` - pipeline parallelism: `api`, `factory`, `infra` (the VM and schedules), and `training`.
+- `model_state/` - checkpoints: `mapper` (graph-based transform engine) and `io` (streaming reader/writers).
+- `dataset/` - distributed-aware dataset wrappers and bucketing.
+- `peft/` - parameter-efficient fine-tuning: `lora`, `full_tune`, `all` (method stacking).
+- `metric/` - distributed-aware metrics: `component` and `impl` (metric catalogue).
+- `optim/` - optimizers, including `stochastic` (stochastic-rounding low-precision).
+- `lr_scheduler/` - learning-rate schedules, including `piecewise` (composable schedules).
+- `tracker/` - experiment tracking integrations (`provider`, e.g. WandB, Aim).
+- `kernel/` - custom kernels: `cce`, `flash_attn`, `gmm`, `moe`, `normalization`, `stochastic`, `swiglu`, `general`.
+- `internals/` - engine internals: `pipeline_state`, `grad_sync`, `grad_norm`, `metric_collector`, `determinism`, `profiling`, `state`.
+
+## Working agreements for agents
+
+- Always run `make lint` before considering a change done. It formats, fixes imports, lints, and type-checks. Type errors are not acceptable in core code.
+- Add tests for any feature or fix. Match the existing tier (`local` vs `distributed`). Note: `make test` includes distributed tests that require an 8-GPU setup; run `make test-local` when GPUs are unavailable, and say so.
+- Follow the Design Principles in CONTRIBUTING.md. PRs that violate them get rejected.
+- PR titles must be Conventional Commits. Versioning is automated via Semantic Release — a wrong `type` produces a wrong release.
+- Do not break public APIs without a DEP. Bug fixes and new models on existing APIs do not need one; breaking changes and new distributed strategies do.
+- Do not add backward-compat shims for old PyTorch/hardware. The project intentionally targets modern APIs (`DTensor`, `DeviceMesh`).
+- Only commit, push, or open PRs when explicitly asked.

CONTRIBUTING.md

@@ -82,8 +82,35 @@ They are not enforced by tooling, but PRs that violate them may be asked to chan
 * **Polymorphism for configurable objects via discriminated unions.** When a configurable object has selectable behavior, model the choices as a Pydantic discriminated union and resolve them in a `build_*()` factory with an exhaustive `match (case _: raise)`.
 
 ### Linting & Formatting
-We use [Ruff](https://docs.astral.sh/ruff/) for both linting and formatting. 
-Configuration is strict (see `pyproject.toml` for enabled rules).
+We use [Ruff](https://docs.astral.sh/ruff/) for both linting and formatting.
+Configuration is strict (see `pyproject.toml` for the authoritative list of enabled rules).
+
+The rule set is broad. The conventions below summarize what it means in practice so you can write
+conforming code without memorizing rule codes.
+
+#### Formatting
+*   Double quotes, 4-space indentation, 120-char line length.
+*   Imports are auto-sorted (`I`). Run `make lint` to fix ordering.
+
+#### Typing & annotations
+*   **Annotate everything public** (`ANN`): function args and return types. `None` returns may be omitted.
+*   `typing.Any` is allowed (`ANN401` is off) but should be a last resort.
+*   Prefer modern syntax (`UP`, `FA`): `X | Y` over `Optional`/`Union`, builtin generics (`list[int]`),
+    and `from __future__ import annotations` where it helps.
+
+#### Naming
+*   `snake_case` for functions/variables, `PascalCase` for classes, `UPPER_CASE` for constants.
+*   Exceptions: `F` (for `nn.functional`) and `BLOCK_SIZE`/uppercase kernel args are allowed.
+*   Private attributes should be prefixed with `_`: `self._something = ...`
+
+#### Boundaries
+*   Every package needs `__init__.py` (`INP`); `/example/` is exempt.
+*   `__init__.py` files expose the package's public surface via an explicit `__all__` re-export list.
+
+#### Tests
+*   Use idiomatic `pytest` (`PT`): `pytest.raises`, fixtures, parametrization.
+*   Tests relax several rules: `assert` is allowed, no docstrings/annotations required, private
+    access and non-top-level imports are fine.
 
 ### Type Checking
 We use **[ty](https://github.com/astral-sh/ty)** to ensure type safety.
@@ -100,6 +127,8 @@ We have two tiers of tests:
 
 **Requirement:** All PRs must pass `make test`. If you add a feature, you must add corresponding tests.
 
+## Documentation
+
 ### Docstrings
 
 We follow the [Google Python style](https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings) for docstrings.
@@ -109,12 +138,13 @@ We follow the [Google Python style](https://google.github.io/styleguide/pyguide.
 *   **Document `__init__`:** Write a docstring even for `__init__`, but keep it short and to the point, e.g. `"""Constructs the ``Trainer`` object."""`.
 *   **Public API coverage:** Always write docstrings for everything considered public API.
 
-## Documentation
+### Documentation Site
+
+The site is built with **Zensical**.
 
-Documentation is built with **Zensical**.
-*   **Docstrings:** Public APIs must have clear docstrings.
-*   **Building:** Run `make mkdocs` to preview changes.
-*   **Location:** Source files are in `docs/` and documented code in `d9d/`.
+*   **Location:** Page sources live in `docs/`; the code they document lives in `d9d/`.
+*   **Building:** Run `make mkdocs` to preview changes locally.
+*   **Registering pages (`zensical.toml`):** The site navigation is **not** auto-generated from the `docs/` directory - it is defined explicitly in the `nav` table of `zensical.toml`. Whenever you add, remove, rename, or move a page under `docs/`, you must update `nav` accordingly. New top-level subsystems should also be added to the appropriate section (and mirrored in `docs/toc.md`).
 
 ## Commit Messages & PRs