Window relation connections on Tortoise, and apply their filter and order (v0.20.0).

Tortoise has no window expression, so it refused eager connections over a
relation and every parent would have cost a query. It now wraps the SQL the
query builder already produced, in its parameterized form so filter values
stay bound rather than pasted into the statement, and the extension awaits an
async backend instead of bailing. Sibling parents share the in-flight read as
a task, since caching a coroutine would leave each of them to start a read of
its own.

A connection's own filter and order were silently ignored on every backend:
the page is cut before the field's resolver runs, so they never reached the
window, and totalCount counted rows the caller had excluded. They are now
converted and applied before the window, and each backend orders by the row
number so a group comes back in the order the window put it in.

CI runs 3.12, 3.13, and 3.14, the versions the classifiers advertise.

Co-authored-by: Cursor <cursoragent@cursor.com>

Author: jamietdavidson

.github/workflows/tests.yml

@@ -11,17 +11,24 @@ permissions:
 
 jobs:
   test:
-    name: Test (Python 3.12)
+    name: Test (Python ${{ matrix.python-version }})
     runs-on: ubuntu-latest
 
+    strategy:
+      fail-fast: false
+      matrix:
+        # Every version here is advertised in the pyproject classifiers; keep
+        # the two lists in step so a claim of support is always backed by a run.
+        python-version: ["3.12", "3.13", "3.14"]
+
     steps:
       - name: Check out repository
         uses: actions/checkout@v4
 
       - name: Set up Python
         uses: actions/setup-python@v5
         with:
-          python-version: "3.12"
+          python-version: ${{ matrix.python-version }}
 
       - name: Set up uv
         uses: astral-sh/setup-uv@v6
@@ -30,15 +37,15 @@ jobs:
 
       - name: Install dependencies
         run: >-
-          uv sync --group dev
+          uv sync --group dev --python ${{ matrix.python-version }}
           --extra django --extra sqlalchemy --extra tortoise
 
       - name: Run tests with coverage
-        run: uv run pytest
+        run: uv run --python ${{ matrix.python-version }} pytest
 
       - name: Upload coverage to Codecov
         uses: codecov/codecov-action@v5
-        if: always()
+        if: always() && matrix.python-version == '3.12'
         with:
           files: coverage.xml
           fail_ci_if_error: false

API_REFERENCE.md

@@ -196,7 +196,7 @@ A Relay connection over a model, split on the same question as fields: does your
 
 `eager` has the library build the query, optionally narrowed by a `(query, info)` scope. The scope applies to `totalCount` and `aggregates` as well as `edges`, since those are computed from the query rather than the returned rows.
 
-On an `@orm.type` an eager connection is served by the parent's relation, and every parent's page is taken in one query using `ROW_NUMBER() OVER (PARTITION BY ...)`, with a second grouped query for the per-parent `totalCount`. That needs a window function and a column on the related rows identifying the parent, so it is refused when the type is defined on Tortoise and for many-to-many relations, both of which point at `lazy`.
+On an `@orm.type` an eager connection is served by the parent's relation, and every parent's page is taken in one query using `ROW_NUMBER() OVER (PARTITION BY ...)`, with a second grouped query for the per-parent `totalCount`. All three backends support this; Tortoise, which has no window expression, wraps its own parameterized SQL to do the numbering. It does need a column on the related rows identifying the parent, so a many-to-many is refused when the type is defined and pointed at `lazy`.
 
 `lazy` takes a resolver receiving `self` that returns the rows, and runs once per parent row.
 

README.md

@@ -197,7 +197,9 @@ class UserNode(relay.Node):
 
 `totalCount` is each parent's own total, counted separately rather than read off the page — the page was truncated, so counting it would report the page size.
 
-This needs a window function and a column on the related rows tying them to a parent, so it is refused when the type is defined if either is missing: on Tortoise, whose query builder has no window functions, and for a many-to-many, whose rows carry the parent key in the association table. Both point you at `orm.connection.lazy`, which is the honest spelling for one query per parent.
+This works on all three backends. Tortoise has no window expression of its own, so that backend wraps the query it built in SQL that does the numbering; the values you filtered on stay bound rather than pasted into the statement.
+
+What it does need is a column on the related rows tying them to a parent, so a many-to-many is refused when the type is defined — those rows keep the parent key in the association table, leaving the window nothing to partition by. The error points you at `orm.connection.lazy`, which is the honest spelling for one query per parent.
 
 Note that a connection on a plain `@strawberry.type` is a *root* connection wherever it appears — it queries the whole table, because nothing about it knows a parent exists. Put connections over relations on an `@orm.type`.
 

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "strawberry-orm"
-version = "0.19.5"
+version = "0.20.0"
 description = "Unified, backend-agnostic ORM abstraction for Strawberry GraphQL"
 readme = "README.md"
 license = "MIT"

src/strawberry_orm/backends/django.py

@@ -551,13 +551,20 @@ def batch_group_items(
             ordering = [F("pk")]
 
         def _run():
-            qs = query.annotate(
-                _rn=Window(
-                    expression=RowNumber(),
-                    partition_by=partition,
-                    order_by=ordering,
+            qs = (
+                query.annotate(
+                    _rn=Window(
+                        expression=RowNumber(),
+                        partition_by=partition,
+                        order_by=ordering,
+                    )
                 )
-            ).filter(_rn__lte=per_group_limit)
+                .filter(_rn__lte=per_group_limit)
+                # Ordered by the row number so each group comes back in the
+                # order the window put it in; without it the rows arrive
+                # however the database found them and the order is lost.
+                .order_by("_rn")
+            )
             rows = list(qs)
             items_by_key: dict[tuple, list[Any]] = defaultdict(list)
             for row in rows:

src/strawberry_orm/backends/sqlalchemy.py

@@ -476,7 +476,10 @@ def batch_group_items(
 
         subq = query.add_columns(rn).subquery()
         ranked_stmt = select(model).from_statement(
-            select(subq).where(subq.c._rn <= per_group_limit)
+            # Ordered by the row number so each group comes back in the order
+            # the window put it in; without it the rows arrive however the
+            # database found them and the requested order is lost.
+            select(subq).where(subq.c._rn <= per_group_limit).order_by(subq.c._rn)
         )
 
         session = self._get_session(info)

src/strawberry_orm/backends/tortoise.py

@@ -30,6 +30,13 @@
     requested_aggregates,
 )
 
+#: Where the window puts each row's number within its group. Dropped again
+#: before the row is turned back into a model, which knows no such column.
+_ROW_NUMBER_ALIAS = "_orm_rn"
+
+#: Where the grouped count puts each group's total.
+_COUNT_ALIAS = "_orm_count"
+
 _TORTOISE_FIELD_MAP: dict[str, type] = {
     "IntField": int,
     "SmallIntField": int,
@@ -669,24 +676,45 @@ async def batch_group_items(
         per_group_limit: int,
         order_input: Any | None = None,
     ) -> dict[tuple, list[Any]]:
-        """Per-group fallback since Tortoise has limited window function support."""
-        groups_qs = await query.group_by(*group_key_fields).values(*group_key_fields)
-        items_by_key: dict[tuple, list[Any]] = {}
+        """Every group's first rows in one windowed query.
+
+        Tortoise has no window expression, so the query it built is wrapped in
+        SQL that numbers rows within each group and keeps the low numbers. The
+        wrapping keeps the placeholders and values Tortoise produced, so filter
+        values are still bound rather than pasted into the statement.
+        """
+        order_clauses = (
+            _build_tortoise_order_from_input(order_input) if order_input else []
+        )
+        sql, values = _windowed_sql(
+            query, model, group_key_fields, order_clauses, per_group_limit
+        )
+        rows = await query._db.execute_query_dict(sql, values)
 
-        for group_dict in groups_qs:
+        items_by_key: dict[tuple, list[Any]] = defaultdict(list)
+        for row in rows:
             key = tuple(
-                str(group_dict[k]) if group_dict[k] is not None else None
+                str(row[k]) if row.get(k) is not None else None
                 for k in group_key_fields
             )
-            scoped = query.filter(**{k: group_dict[k] for k in group_key_fields})
-            if order_input:
-                order_clauses = _build_tortoise_order_from_input(order_input)
-                if order_clauses:
-                    scoped = scoped.order_by(*order_clauses)
-            items = await scoped.limit(per_group_limit)
-            items_by_key[key] = list(items)
-
-        return items_by_key
+            row.pop(_ROW_NUMBER_ALIAS, None)
+            items_by_key[key].append(model._init_from_db(**row))
+        return dict(items_by_key)
+
+    async def group_counts(
+        self, query: Any, key_field: str, info: Any
+    ) -> dict[Any, int]:
+        """Each group's real total, which the windowed page cannot report."""
+        from tortoise.functions import Count
+
+        # Inherited ordering would drag non-grouped columns into the statement.
+        counted = (
+            query.order_by()
+            .group_by(key_field)
+            .annotate(**{_COUNT_ALIAS: Count(query.model._meta.pk_attr)})
+            .values(key_field, _COUNT_ALIAS)
+        )
+        return {row[key_field]: row[_COUNT_ALIAS] for row in await counted}
 
     # -- Queryset overrides --------------------------------------------------
 
@@ -760,9 +788,64 @@ async def materialize_query(self, query: Any, info: Any) -> list[Any]:
 
     # -- Optimizer -----------------------------------------------------------
 
+    _supports_windowed_pages = True
+
     def optimizer_extension(self, **kwargs: Any) -> type[SchemaExtension]:
         return OptimizerExtension.configure(backend=self, store=self._store)
 
+    def instance_pk(self, instance: Any) -> Any:
+        return getattr(instance, "pk", None)
+
+    def _relation_connection_spec(
+        self, model: type, field_name: str, relation: str
+    ) -> Any:
+        from tortoise.fields.relational import BackwardFKRelation
+
+        from strawberry_orm.backends._base import RelationConnectionSpec
+
+        field = model._meta.fields_map.get(relation)  # type: ignore[attr-defined]
+        key_field = getattr(field, "relation_field", None)
+        # Only a reverse foreign key keeps the parent's key on the related row.
+        # A many-to-many hides it in the through table, leaving the window
+        # nothing to partition by.
+        if not isinstance(field, BackwardFKRelation) or key_field is None:
+            return None
+        return RelationConnectionSpec(
+            model=model,
+            field_name=field_name,
+            relation=relation,
+            related_model=self._relation_target_model(model, relation),
+            key_field=key_field,
+        )
+
+    def relation_base_query(self, spec: Any, pks: list[Any], info: Any) -> Any:
+        qs = spec.related_model.filter(**{f"{spec.key_field}__in": pks})
+        restrict = self.relation_scope(
+            spec.model, spec.field_name, info, on=spec.relation
+        )
+        return qs if restrict is None else restrict(qs, info)
+
+    def _make_relation_query_resolver(
+        self, model: type, field_name: str, relation: str
+    ) -> Any:
+        backend = self
+        spec = self._relation_connection_spec(model, field_name, relation)
+
+        def resolver(self: Any, info: Any) -> Any:
+            from strawberry_orm.batching import page_attr
+
+            page = getattr(self, page_attr(field_name), None)
+            if page is not None:
+                return page
+            qs = spec.related_model.filter(
+                **{spec.key_field: backend.instance_pk(self)}
+            )
+            restrict = backend.relation_scope(model, field_name, info, on=relation)
+            return qs if restrict is None else restrict(qs, info)
+
+        resolver.__name__ = field_name
+        return resolver
+
     def _apply_nested_queryset(
         self,
         qs: Any,
@@ -886,6 +969,13 @@ def _walk_selections(
                 if not is_rel:
                     continue
 
+                # A field that answers for itself will ignore whatever the
+                # prefetch loads, so do not pay for it.
+                if self.resolves_itself(
+                    self._type_name_for_model(current_model), field_name
+                ):
+                    continue
+
                 related_model = field_obj.related_model
 
                 ancestor = _find_custom_ancestor(full_path)
@@ -1795,6 +1885,84 @@ def _extract_tortoise_overlapping_order(
     return clauses
 
 
+def _db_columns(model: type) -> dict[str, str]:
+    """Field name to column name, for every column the model really has."""
+    return dict(model._meta.fields_db_projection)  # type: ignore[attr-defined]
+
+
+def _quote_ident(name: str, model: type, quote_char: str) -> str:
+    """Quote *name* for the dialect, having checked the model owns it.
+
+    Only ever called with names the caller derived from the model, so an
+    unknown one means a bug rather than user input; refusing it anyway keeps
+    the window's SQL from being assembled out of anything but real columns.
+    """
+    columns = set(_db_columns(model).values())
+    if name not in columns:
+        raise ValueError(
+            f"{model.__name__} has no column {name!r} to build a window from."
+        )
+    return f"{quote_char}{name}{quote_char}"
+
+
+def _window_ordering(
+    model: type, order_clauses: list[str], quote_char: str
+) -> list[str]:
+    """Render the window's ORDER BY, falling back to the primary key."""
+    meta = model._meta  # type: ignore[attr-defined]
+    projection = _db_columns(model)
+    rendered: list[str] = []
+    for clause in order_clauses:
+        descending = clause.startswith("-")
+        field = clause[1:] if descending else clause
+        column = projection.get(field, field)
+        direction = " DESC" if descending else " ASC"
+        rendered.append(_quote_ident(column, model, quote_char) + direction)
+    if not rendered:
+        pk_column = projection.get(meta.pk_attr, meta.db_pk_column)
+        rendered.append(_quote_ident(pk_column, model, quote_char) + " ASC")
+    return rendered
+
+
+def _windowed_sql(
+    query: Any,
+    model: type,
+    group_key_fields: list[str],
+    order_clauses: list[str],
+    per_group_limit: int,
+) -> tuple[str, list[Any]]:
+    """Wrap *query* in SQL keeping the first rows of every group.
+
+    Returns the statement and the values its placeholders still expect, so the
+    caller binds them rather than embedding them.
+    """
+    query._choose_db_if_not_chosen()
+    query._make_query()
+    inner, values = query.query.get_parameterized_sql()
+
+    quote_char = query._db.query_class.SQL_CONTEXT.quote_char
+    projection = _db_columns(model)
+    partition = ", ".join(
+        _quote_ident(projection.get(field, field), model, quote_char)
+        for field in group_key_fields
+    )
+    ordering = ", ".join(_window_ordering(model, order_clauses, quote_char))
+    alias = f"{quote_char}{_ROW_NUMBER_ALIAS}{quote_char}"
+
+    sql = (
+        f"SELECT * FROM ("
+        f"SELECT _orm_inner.*, ROW_NUMBER() OVER ("
+        f"PARTITION BY {partition} ORDER BY {ordering}"
+        f") AS {alias} FROM ({inner}) _orm_inner"
+        f") _orm_windowed WHERE {alias} <= {int(per_group_limit)} "
+        # Ordered by the row number so each group comes back in the order the
+        # window put it in; without it the rows arrive however the database
+        # found them and the requested order is lost.
+        f"ORDER BY {alias}"
+    )
+    return sql, values
+
+
 def _build_tortoise_order_from_input(order_input: Any) -> list[str]:
     """Convert an order input to Tortoise order_by strings."""
     order_list = order_input if isinstance(order_input, list) else [order_input]