Skip to content

Commit de6d79f

Browse files
lesnik512lesnik512
authored
feat: add transactional inbox ledger (#1)
* feat: add transactional inbox ledger Inbox(session, table) records processed messages by (scope, message_id) in the caller's transaction, with database-clock cleanup in bounded batches. Tests run against PostgreSQL 17 and cover concurrent claims, rollback, cancellation, commit failure and cleanup. * fix: raise SQLAlchemy floor below Python 3.13 to 2.0.20 db-retry 0.9, a dev dependency, requires sqlalchemy>=2.0.20, so the floors job could not install 2.0.0 and that floor was never tested.
1 parent 50d8944 commit de6d79f

25 files changed

Lines changed: 1526 additions & 0 deletions

‎.dockerignore‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
.env
2+
.coverage
3+
.gitignore
4+
.idea
5+
.ruff_cache
6+
.vscode
7+
.git
8+
.pytest_cache
9+
.DS_Store
10+
*.yml
11+
Dockerfile
12+
**/__pycache__
13+
.hypothesis
14+
.venv

‎.github/workflows/_checks.yml‎

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
name: checks
2+
3+
on:
4+
workflow_call: {}
5+
6+
jobs:
7+
lint:
8+
runs-on: ubuntu-latest
9+
steps:
10+
- uses: actions/checkout@v6
11+
- uses: extractions/setup-just@v4
12+
- uses: astral-sh/setup-uv@v8.2.0
13+
with:
14+
enable-cache: true
15+
cache-dependency-glob: "**/pyproject.toml"
16+
- run: uv python install 3.11
17+
- run: uv python pin 3.11
18+
- run: just install lint-ci
19+
20+
links:
21+
runs-on: ubuntu-latest
22+
steps:
23+
- uses: actions/checkout@v6
24+
- name: Check local links
25+
uses: lycheeverse/lychee-action@v2
26+
with:
27+
args: >-
28+
--offline
29+
--no-progress
30+
--remap
31+
'https://github\.com/${{ github.repository }}/(?:blob|tree)/main/(.*) file://${{ github.workspace }}/$1'
32+
'**/*.md'
33+
34+
pytest:
35+
runs-on: ubuntu-latest
36+
strategy:
37+
fail-fast: false
38+
matrix:
39+
python-version:
40+
- "3.11"
41+
- "3.12"
42+
- "3.13"
43+
- "3.14"
44+
- "3.14t"
45+
- "3.15"
46+
- "3.15t"
47+
services:
48+
postgres:
49+
image: postgres:17
50+
env:
51+
POSTGRES_USER: inbox
52+
POSTGRES_PASSWORD: inbox
53+
POSTGRES_DB: inbox
54+
ports:
55+
- 5432:5432
56+
options: >-
57+
--health-cmd "pg_isready -U inbox"
58+
--health-interval 10s
59+
--health-timeout 5s
60+
--health-retries 5
61+
steps:
62+
- uses: actions/checkout@v6
63+
- uses: extractions/setup-just@v4
64+
- uses: astral-sh/setup-uv@v8.2.0
65+
with:
66+
enable-cache: true
67+
cache-dependency-glob: "**/pyproject.toml"
68+
- run: uv python install ${{ matrix.python-version }}
69+
- run: uv python pin ${{ matrix.python-version }}
70+
- run: just install
71+
- name: Assert the GIL is actually disabled
72+
if: endsWith(matrix.python-version, 't')
73+
run: uv run --no-sync python -c "import sys, asyncpg, sqlalchemy, sqlalchemy_inbox; assert not sys._is_gil_enabled(), 'GIL re-enabled - a dependency lacks free-threaded support'"
74+
- run: just test-ci
75+
env:
76+
PYTHONDONTWRITEBYTECODE: 1
77+
PYTHONUNBUFFERED: 1
78+
POSTGRES_DSN: postgresql+asyncpg://inbox:inbox@127.0.0.1:5432/inbox
79+
80+
floors:
81+
# Installs the declared floors on each Python, against the oldest supported PostgreSQL.
82+
if: github.event_name != 'schedule'
83+
runs-on: ubuntu-latest
84+
strategy:
85+
fail-fast: false
86+
matrix:
87+
python-version:
88+
- "3.11"
89+
- "3.12"
90+
- "3.13"
91+
- "3.14"
92+
services:
93+
postgres:
94+
image: postgres:14
95+
env:
96+
POSTGRES_USER: inbox
97+
POSTGRES_PASSWORD: inbox
98+
POSTGRES_DB: inbox
99+
ports:
100+
- 5432:5432
101+
options: >-
102+
--health-cmd "pg_isready -U inbox"
103+
--health-interval 10s
104+
--health-timeout 5s
105+
--health-retries 5
106+
steps:
107+
- uses: actions/checkout@v6
108+
- uses: astral-sh/setup-uv@v8.2.0
109+
with:
110+
enable-cache: true
111+
cache-dependency-glob: "**/pyproject.toml"
112+
- run: uv python install ${{ matrix.python-version }}
113+
- run: uv python pin ${{ matrix.python-version }}
114+
- run: uv venv
115+
- if: matrix.python-version == '3.11'
116+
run: uv pip compile pyproject.toml --no-deps --resolution lowest-direct -o floors.txt
117+
- if: matrix.python-version != '3.11'
118+
run: uv pip compile pyproject.toml --resolution lowest-direct --no-build -o floors.txt
119+
- run: uv pip install -r pyproject.toml --group dev --constraints floors.txt --no-build
120+
- run: uv run --no-sync pytest . --no-cov
121+
env:
122+
PYTHONDONTWRITEBYTECODE: 1
123+
PYTHONUNBUFFERED: 1
124+
POSTGRES_DSN: postgresql+asyncpg://inbox:inbox@127.0.0.1:5432/inbox

‎.github/workflows/ci.yml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
name: main
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
pull_request: {}
8+
9+
concurrency:
10+
group: ${{ github.head_ref || github.run_id }}
11+
cancel-in-progress: true
12+
13+
jobs:
14+
checks:
15+
uses: ./.github/workflows/_checks.yml

‎.github/workflows/release.yml‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
name: Release
2+
3+
# Tag-driven: pushing a semver tag publishes to PyPI and creates the matching
4+
# GitHub Release. Replaces the old `on: release: published` publish.yml — that
5+
# trigger is removed so the published Release this workflow creates can't re-fire
6+
# it (double-publish). The tag is the sole entry point; by convention a tag is
7+
# only cut off a green main, so there is no in-workflow CI gate.
8+
on:
9+
push:
10+
tags:
11+
- '[0-9]+.[0-9]+.[0-9]+' # stable: 2.7.2
12+
- '[0-9]+.[0-9]+.[0-9]+[a-z]+[0-9]+' # pre-release: 2.0.0rc1, 4.0.0a2
13+
14+
# contents: write -> create the GitHub Release; id-token: write -> OIDC for PyPI Trusted Publishing.
15+
permissions:
16+
contents: write
17+
id-token: write
18+
19+
jobs:
20+
release:
21+
runs-on: ubuntu-latest
22+
environment: pypi # scopes the PyPI Trusted Publisher; hook for approval rules
23+
steps:
24+
- uses: actions/checkout@v6
25+
- uses: extractions/setup-just@v4
26+
- uses: astral-sh/setup-uv@v7
27+
28+
# PyPI is irreversible, so it runs FIRST: if it fails the job stops and no
29+
# GitHub Release is created advertising a version that never reached PyPI.
30+
# `just publish` derives the version from $GITHUB_REF_NAME (the tag name).
31+
# Auth via PyPI Trusted Publishing (OIDC); no PYPI_TOKEN. Needs a Trusted
32+
# Publisher on the sqlalchemy-inbox PyPI project (env: pypi, workflow: release.yml).
33+
- run: just publish
34+
35+
# The Release body is GitHub's generated notes, rendered from the squashed
36+
# PR titles since the previous tag — so a conventional-commit title is what
37+
# a reader gets. A release wanting prose is edited after the fact with
38+
# `gh release edit <tag> --notes-file`. A tag with a letter (2.0.0rc1) is a
39+
# pre-release -> flagged so GitHub won't mark it "Latest".
40+
- name: Resolve release metadata
41+
id: meta
42+
run: |
43+
set -euo pipefail
44+
if [[ "$GITHUB_REF_NAME" =~ [a-z] ]]; then
45+
echo "prerelease=true" >> "$GITHUB_OUTPUT"
46+
else
47+
echo "prerelease=false" >> "$GITHUB_OUTPUT"
48+
fi
49+
50+
- name: Publish GitHub Release
51+
uses: softprops/action-gh-release@v3
52+
with:
53+
generate_release_notes: true
54+
prerelease: ${{ steps.meta.outputs.prerelease }}
55+
draft: false

‎.github/workflows/scheduled.yml‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
name: scheduled-dep-check
2+
on:
3+
schedule:
4+
- cron: "0 6 * * *" # Daily 06:00 UTC
5+
workflow_dispatch: {}
6+
7+
concurrency:
8+
group: scheduled-dep-check
9+
cancel-in-progress: false
10+
11+
jobs:
12+
checks:
13+
uses: ./.github/workflows/_checks.yml
14+
15+
report-failure:
16+
needs: checks
17+
if: failure() && github.event_name == 'schedule'
18+
permissions:
19+
issues: write
20+
uses: modern-python/.github/.github/workflows/report-scheduled-failure.yml@main

‎.gitignore‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Generic things
2+
*.pyc
3+
*~
4+
__pycache__/*
5+
*.swp
6+
*.sqlite3
7+
*.map
8+
.vscode
9+
.idea
10+
.DS_Store
11+
.env
12+
.pytest_cache
13+
.ruff_cache
14+
.coverage
15+
htmlcov/
16+
coverage.xml
17+
pytest.xml
18+
dist/
19+
.python-version
20+
.venv
21+
uv.lock
22+
/site/

‎AGENTS.md‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# AGENTS.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Project Overview
6+
7+
`sqlalchemy-inbox` is a transactional inbox ledger for SQLAlchemy asyncio on PostgreSQL.
8+
[`CONTEXT.md`](CONTEXT.md) opens with what it does and owns the vocabulary. Read it before naming a
9+
concept in code, a test name, or an issue title.
10+
11+
## Commands
12+
13+
`just` (task runner) and `uv` (package manager). The [`justfile`](justfile) is the source of truth.
14+
The things it does not say:
15+
16+
- `just test [args]` forwards args **unquoted**, so a spaced `-k` expression word-splits and fails.
17+
- Every test needs PostgreSQL and skips if `POSTGRES_DSN` is unreachable. The compose Postgres
18+
publishes no host port, so run tests through `just test`. The coverage gate is on in CI; pass
19+
`--no-cov` while iterating locally if you add `--cov`.
20+
21+
## Architecture
22+
23+
- `Inbox` never commits, rolls back, begins, or opens a session or connection. Transaction
24+
ownership stays with the caller. Any new method must keep that.
25+
- `Inbox.claim` requires an open transaction on its session. That is what turns a wrongly wired
26+
second session into an error instead of a marker that never commits.
27+
- Concurrency claims are proven only by tests on separate connections, never one shared test
28+
transaction.
29+
30+
## Workflow
31+
32+
Every link in `README.md` must be absolute: `https://github.com/modern-python/sqlalchemy-inbox/blob/main/<path>`.
33+
`README.md` is also the PyPI long description, and PyPI does not rewrite relative links.
34+
35+
## Code Style
36+
37+
- **Never use local/inline imports**, tests included. `ruff` catches them.
38+
- Annotate every function argument.
39+
- Docstrings: public API documents the contract; internal helpers get a one-line contract at most.

‎CLAUDE.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
@AGENTS.md

‎CONTEXT.md‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# sqlalchemy-inbox
2+
3+
A processed-message ledger for consumers on SQLAlchemy's asyncio extension and PostgreSQL: one row
4+
per (scope, message id), inserted in the caller's transaction so it commits or rolls back with the
5+
business writes. Postgres-only. It stores no payload and runs no worker.
6+
7+
## Language
8+
9+
A term is listed only when there is a synonym to reject, or a meaning subtle enough that code and
10+
docs must agree on it. General programming vocabulary does not belong here, however heavily this
11+
project uses it.
12+
13+
**Marker**:
14+
One ledger row, keyed by `(scope, message_id)`. Its presence after commit means the database
15+
effects for that key have committed.
16+
_Avoid_: lock, processed event, dedup record
17+
18+
**Scope**:
19+
A stable string naming one logical consumer of a message, such as `chats.create-chat.v1`. Two
20+
consumers that process the same message independently use different scopes.
21+
_Avoid_: handler name, consumer group (both change on refactors and deployment renames)
22+
23+
**Message id**:
24+
The producer-assigned identity of a message, the same on every delivery.
25+
_Avoid_: offset, delivery id (both change between deliveries)
26+
27+
**Claim**:
28+
Inserting a marker in the caller's open transaction. A claim that inserted returns `True`; one
29+
that found a committed marker returns `False`.
30+
_Avoid_: acquire, lock, register
31+
32+
**Retention**:
33+
How long a marker is kept. After it is deleted, the same key is new again.
34+
_Avoid_: TTL (nothing expires on its own; cleanup deletes)
35+
36+
**Cutoff**:
37+
The database timestamp, taken once per cleanup run, before which markers are deleted.
38+
_Avoid_: deadline, expiry time

‎Dockerfile‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
FROM python:3.14-slim
2+
3+
COPY --from=ghcr.io/astral-sh/uv:latest /uv /bin/uv
4+
RUN useradd --no-create-home --gid root runner
5+
6+
ENV UV_PROJECT_ENVIRONMENT=/code/.venv \
7+
UV_NO_MANAGED_PYTHON=1 \
8+
UV_NO_CACHE=true \
9+
UV_LINK_MODE=copy
10+
11+
WORKDIR /code
12+
13+
COPY pyproject.toml .
14+
COPY uv.lock .
15+
16+
RUN uv sync --all-extras --frozen --no-install-project
17+
18+
COPY . .
19+
20+
RUN chown -R runner:root /code && chmod -R g=u /code
21+
22+
USER runner
23+
24+
RUN uv sync --all-extras --frozen

0 commit comments

Comments
 (0)