Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions docs/best-practices.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,29 @@ single response.
This will help to reduce the response time and the load on the server, and improve the performance of your
application.

### Load relations for a whole result set, not per record

Touching a relation on each record of a result set issues one query per record — the N+1 problem. Name
the relation on the query instead, and it is resolved for the entire batch in a constant number of
queries:

```cpp
auto albums = dm.Query<Album>().With<&Album::tracks>().All(); // 2 queries, not 1 + N
```

A nested relation needs the whole path named — `.With<&Track::album, &Album::artist>()` — because
each record holds its own copy of the target, so one level of eager loading leaves the level below it
loading per record. `DataMapperOptions { .eagerLoadDepth = N }` loads everything reachable instead,
at the cost of fetching more than you asked for.

See [Eager loading of relations](usage.md). Two things compound with it:

- **Index your foreign keys.** `CreateTable<Record>()` emits an index for every `BelongsTo` column,
because no supported engine indexes a foreign key implicitly. Tables created by hand, or by an older
version of Lightweight, need that index added — without it every relation query is a full table scan.
- **Prove the absence of N+1 in tests.** A `SqlLogger` subclass counting `OnPrepare`/`OnExecuteDirect`
turns "this endpoint issues two queries" into an assertion instead of an assumption.

### Let block-prefetch cut network round-trips

Per-row fetch loops issue one `SQLFetch` (one network round-trip) per row. Lightweight transparently
Expand Down
81 changes: 81 additions & 0 deletions docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,87 @@ void BulkInsert(DataMapper& dm, std::vector<Person> const& people)
> records (treat them as write-only inputs), and `UpdateAll` writes a uniform set of columns for every
> row rather than only the per-record modified ones. The range must be contiguous.

### Eager loading of relations (`With<>()`)

Accessing a relation on a query result loads it on demand — one query per record. Over a result set of
N records that is the N+1 problem: reading `album.tracks` for 1000 albums issues 1001 queries.
`With<&Record::relation>()` instead resolves the relation for the whole result set once it has been
materialized, using `WHERE <key> IN (...)`:

```cpp
// Two queries in total, whatever the number of albums: one for the albums, one for all their tracks.
auto albums = dm.Query<Album>()
.With<&Album::tracks>() // HasMany
.With<&Album::artist>() // BelongsTo
.All();

for (auto& album: albums)
for (auto const& track: album.tracks.All()) // already loaded, no query
std::println("{} - {}", album.title, track->title);
```

- Supported for `BelongsTo` and `HasMany`. `HasOneThrough`, `HasManyThrough` and `CompositeForeignKey`
still load on demand; naming one of them in `With<>()` is a compile error rather than a silent
fallback.
- Naming several relations forms a **path**, which is what a nested relation needs — see below.
- Calls chain, one per relation to load. It applies to `All()`, `First()`, `First(n)` and `Range()`.
- The `IN` predicate is chunked (see `SqlQueryFormatter::MaxInPredicateValues`, 1000 by default), so a
large batch costs one query per chunk — a constant number of queries per relation, never one per
record.
- A `BelongsTo` whose foreign key is `NULL`, and an owner with no children, are handled without an
extra query: the childless owner's relation is marked loaded-and-empty rather than left to query for
a result already known.
- Relations that were *not* named keep their on-demand behaviour. Combining `With<>()` with
`DataMapperOptions { .loadRelations = false }` therefore turns any unrequested relation access into a
`SqlRequireLoadedError` instead of a silent query — useful to prove a code path issues no N+1.

#### Nested relations

Eager-loading one level is not enough for a chain. Every record holds its *own copy* of its
`BelongsTo` target, so reaching a relation of that copy runs the copy's own lazy loader — the N+1
simply moves one level down. Name the whole path instead:

```cpp
auto tracks = dm.Query<Track>()
.With<&Track::album>() // 1 query for all albums
.With<&Track::album, &Album::artist>() // 1 query for all those albums' artists
.All();

for (auto& track: tracks)
std::println("{} - {}", track.album.Record().title,
track.album.Record().artist.Record().name); // no queries here
```

Three queries in total, for any number of tracks. Each level is resolved for every record reached by
the level above it, at once. A path may also run through the "many" side
(`.With<&Album::tracks, &Track::genre>()`): the middle level fans out, and the level below it is
still one query rather than one per child.

Already-loaded relations are skipped, so overlapping paths (`.With<&A::b>()` next to
`.With<&A::b, &B::c>()`) do not fetch `b` twice.

#### Loading everything reachable

When a whole object graph is wanted rather than named paths, set a depth on the query instead:

```cpp
// Tracks, their albums and categories, and those albums' artists - a constant number of queries.
auto tracks = dm.Query<Track, DataMapperOptions { .eagerLoadDepth = 2 }>().All();
```

`eagerLoadDepth` batch-loads *every* `BelongsTo` and `HasMany` reachable within that many levels.
Prefer `With<>()` when only part of the graph is needed: the depth walk fetches more rows, and
instantiates the loader for the whole reachable relation graph, which costs compile time. The depth
is what bounds both — and what lets a cyclic graph (a self-referencing record, or `A → B → A`)
terminate, since the recursion is cut at a compile-time constant.

Measured on 1000 owners with 10 children each, comparing the on-demand path with `With<>()`:

| relation | queries before | queries after | SQLite3 | PostgreSQL | MS SQL Server |
|---|---:|---:|---:|---:|---:|
| `HasMany` | 1001 | 2 | 8.7x | 45x | 45x |
| `BelongsTo` | 10001 | 2 | 37x | 464x | 407x |

## Simple row retrieval via structs

When only read access is needed, you can use a simple `struct` to represent the row,
Expand Down
18 changes: 18 additions & 0 deletions src/Lightweight/DataMapper/BelongsTo.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -316,6 +316,24 @@ class BelongsTo
return static_cast<bool>(_referencedFieldValue);
}

/// @brief Returns the already-loaded referenced record, or `nullptr` when none is loaded.
///
/// Unlike `Record()`, this never runs the on-demand loader: it reports what is present right
/// now. That is what lets the batched relation loading walk one level deeper (`With<A, B>()`)
/// without turning the walk itself into the N+1 it exists to remove.
///
/// @return Pointer to the loaded record, or `nullptr` if the relation is unloaded or NULL.
[[nodiscard]] LIGHTWEIGHT_FORCE_INLINE constexpr ReferencedRecord* LoadedRecord() noexcept
{
return _record.get();
}

/// @copydoc LoadedRecord()
[[nodiscard]] LIGHTWEIGHT_FORCE_INLINE constexpr ReferencedRecord const* LoadedRecord() const noexcept
{
return _record.get();
}

/// Emplaces a record into the relationship. This will mark the relationship as loaded.
[[nodiscard]] LIGHTWEIGHT_FORCE_INLINE constexpr ReferencedRecord& EmplaceRecord()
{
Expand Down
Loading
Loading