Skip to content

Commit 9401ccd

Browse files
author
Gurov Oleg
committed
feat: release v0.7.2 with reusable datasets and guarded benchmark runs
Add each-iteration/once/skip initialization policies, server-wide reset coordination, bounded SQL workloads and partial-result preservation. Improve OS sampling and initialization evidence. Align IMDb and Pagila generators with workload semantics, retain planner diagnostics, and update usage documentation and regression coverage.
1 parent e0ca7d5 commit 9401ccd

241 files changed

Lines changed: 9127 additions & 4993 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎INITIALIZATION.md‎

Lines changed: 33 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -10,9 +10,10 @@ selects it. Arbitrary old commands remain supported without SQL rewriting.
1010

1111
| Option | Default | Meaning |
1212
| --- | --- | --- |
13+
| `--init-policy` | `each-iteration` | Reset/load every iteration, only the first (`once`), or use existing data (`skip`) |
1314
| `--init-workers` | `4` | Maximum simultaneous data or index jobs; each owns its connection |
1415
| `--init-batch-rows` | `100000` | Target rows per committed data batch |
15-
| `--init-table-mode` | `unlogged` | Stored tables become UNLOGGED before data, then LOGGED afterward; `logged` skips conversion |
16+
| `--init-table-mode` | `unlogged` | `unlogged`: SET UNLOGGED before loading, SET LOGGED afterward; `logged`: SET LOGGED before loading, no post-load conversion |
1617
| `--init-fsync` | `off` | Temporarily disable fsync on the selected primary; `keep` preserves it |
1718
| `--init-synchronous-commit` | `keep` | Preserve the session commit policy; explicit `off` or `local` opts into faster loading |
1819

@@ -31,7 +32,7 @@ be added to elapsed stage times. DDL and maintenance details are collapsed and
3132
do not display artificial zero row counters. A task targeting a partitioned
3233
parent can populate several partitions without separate partition-level jobs.
3334

34-
Before each iteration, `--reset-mode database` (default) recreates the disposable
35+
When initialization is scheduled, `--reset-mode database` (default) recreates the disposable
3536
database. `--reset-mode schema` resets the schemas in `LoadPlan.schemas` inside
3637
a pre-created dedicated database; it requires fast initialization and `--init-fsync keep`.
3738
It uses the target database for version checks, the controller connection, reset,
@@ -42,11 +43,37 @@ an optional `--managed-pg-info FILE` embeds provider metadata and also implies t
4243
Schema reset checks CREATE privilege, schema ownership, read-write status and
4344
UNLOGGED/LOGGED conversion before deleting existing schemas. It refuses `public`,
4445
system schemas and system databases. A session advisory lock is held through the
45-
workload to prevent another schema-mode run from resetting the same database.
46+
workload. A cross-database lock check also rejects a competing full database reset,
47+
whose controller is connected to `postgres`.
4648
Direct connections and compatible session pooling are supported; transaction/statement pooling
4749
are not. See the [pool requirements](doc/managed_mode_usage.md#connection-pooling). Reset does not alter database ownership, ACLs or permanent role settings.
4850
Use a dedicated database: CASCADE also removes objects depending on the profile schemas.
4951

52+
`--init-policy each-iteration` preserves the default behavior. `once` runs the
53+
following preparation only before the first point. `skip` runs no preparation,
54+
including no initialization command or vacuum/analyze, and does not require
55+
`--allow-database-reset`. It ignores loader settings and the reset scope, rejects
56+
custom initialization commands/server configurations/cache dropping, and uses the
57+
target database for the controller and version check. Pending fsync recovery must
58+
be resolved before `skip`; it never silently restores server settings.
59+
60+
The `once` and `skip` policies retain a controller advisory lock across the
61+
client sweep and release it on completion, failure or cancellation. The controller
62+
uses the target database for schema reset/skip, or `postgres` for database reset.
63+
All modes check the lock key across the connected server's databases before any
64+
reset or custom configuration write; this also covers legacy initialization.
65+
Different databases on the same server cannot run overlapping measurements.
66+
After a planned full-reset restart, the lock must be reacquired before creating
67+
the target database. Unexpected connection loss does not trigger automatic resume.
68+
See [concurrent runs and reset protection](README.md#concurrent-runs-and-reset-protection)
69+
for connection requirements and limits.
70+
71+
Reused points inherit cache state and any workload writes from earlier points. Initialization policy and
72+
per-point initialization status are recorded in reports. For the structural
73+
checks and their limits, see [initialization policies](README.md#bundled-workload-profiles).
74+
`--drop-os-caches` requires `each-iteration`; `once` may apply a custom server
75+
configuration only as part of its first database reset.
76+
5077
Order for each fresh dataset:
5178

5279
1. Create the schema without indexes or foreign keys; make stored tables UNLOGGED.
@@ -65,8 +92,9 @@ Order for each fresh dataset:
6592
host command, and replica wait. It does not bound the sum of all data batches.
6693
For large datasets, increase it to cover the longest table rewrite/index build
6794
and the time replicas need to catch up. A timeout or failed phase aborts the run
68-
before pgbench. Failed batches are not automatically retried; the next run
69-
resets the database or profile schemas according to `--reset-mode`.
95+
before pgbench. Failed batches are not automatically retried; the next initializing run
96+
resets the database or profile schemas according to `--reset-mode`. Do not use
97+
`skip` on a partially initialized dataset.
7098

7199
The common loader sets `search_path` from `LoadPlan.schemas` through SQL after
72100
opening each asyncpg connection. Explicit loader commit overrides are applied and

0 commit comments

Comments
 (0)