@@ -66,6 +66,7 @@ selected target. The workload commands themselves run on the machine where
6666| ` local ` | local machine | local machine |
6767| ` docker ` | existing container | local machine through a published port |
6868| ` ssh ` | remote host | local machine through an SSH local-forwarding port |
69+ | ` --managed-pg-info FILE ` | PostgreSQL protocol only; no host transport | local machine through the managed endpoint |
6970
7071This separation keeps workload generation independent of target management and
7172makes the measured client location explicit.
@@ -253,8 +254,9 @@ commands. It must be longer than the expected command duration.
253254| ` ARG_PG_DATABASE ` | ` --database ` |
254255| ` ARG_PGBENCH_PATH ` | newest local pgbench, or validated ` --pgbench-path ` |
255256| ` ARG_PSQL_PATH ` | matching local psql, or validated ` --psql-path ` |
256- | ` ARG_WORKLOAD_PATH ` | ` --workload-path ` |
257+ | ` ARG_WORKLOAD_PATH ` | bundled profile directory, or ` --workload-path ` |
257258| ` ARG_WORKLOAD_SCALE ` | ` --workload-scale ` |
259+ | ` ARG_WORKLOAD_DURATION_SECONDS ` | ` --workload-duration-seconds ` , or the profile's ` default_duration_seconds ` |
258260| ` ARG_PGBENCH_CLIENTS ` | current client-axis value |
259261| ` ARG_PGBENCH_TIME ` | current duration-axis value |
260262
@@ -265,16 +267,228 @@ For a custom workload, use `--benchmark-type custom`, supply an existing
265267` --workload-path ` , and reference files below that path from the command
266268templates.
267269
268- For a packaged maximum-TPS workload, use ` --workload-profile imdb ` or
269- ` --workload-profile pagila ` , ` --workload-scale SCALE ` , and a
270- ` --pgbench-clients ` sweep. ` pg-perf-bench profiles ` lists the installed
271- collection. Each profile supplies its schema, deterministic Python generator,
272- typical SQL query set and command templates. It deliberately does not reuse
273- ` pg_workload ` 's scheduler-specific ` profile.yml ` .
270+ ### Bundled workload profiles
271+
272+ ` pg-perf-bench profiles ` lists the installed profiles, their default scale,
273+ duration and script count. Select one with ` --workload-profile ` ; its schema,
274+ generator, setup and workload commands are supplied automatically.
275+
276+ | Profile | Workload and default script weights | Default duration |
277+ | ---| ---| ---:|
278+ | [ ` imdb ` ] ( src/pg_perf_bench/workload_profiles/imdb/README.md ) | 38 analytical scripts over 21 tables; equal weights | 120 s |
279+ | [ ` pagila ` ] ( src/pg_perf_bench/workload_profiles/pagila/README.md ) | OLTP: select / insert / update / delete = 50 / 25 / 20 / 5 | 60 s |
280+ | [ ` pagila-htap ` ] ( src/pg_perf_bench/workload_profiles/pagila-htap/README.md ) | The same OLTP scripts plus reporting: 50 / 25 / 20 / 5 / 5 | 60 s |
281+
282+ Weights describe selection of whole pgbench scripts, each of which can execute
283+ several SQL statements or transactions. The default HTAP reporting share is
284+ ` 5 / 105 ` , approximately 4.8 %. Its reported TPS includes all five scripts.
285+
286+ | Setting | How to configure it |
287+ | ---| ---|
288+ | Data volume | ` --workload-scale SCALE ` , default ` 1 ` ; positive fractional values such as ` 0.25 ` are accepted. Generators retain minimum table sizes at small scales. |
289+ | Concurrent clients | ` --pgbench-clients 1,2,4,8,16 ` ; each value gets a freshly recreated database. Bundled commands also use one pgbench job per client. |
290+ | Measured window per point | ` --workload-duration-seconds 120 ` ; overrides the profile default for every client count. |
291+ | Command time limit | ` --command-timeout 300 ` ; allow enough time for initialization and for the workload window plus completion of in-flight queries. |
292+
293+ Bundled profiles require ` --pgbench-clients ` ; ` --pgbench-time ` is rejected.
294+ They select benchmark type ` custom ` automatically. ` --workload-path ` cannot be
295+ combined with ` --workload-profile ` .
296+
297+ For example, run the HTAP profile on a dedicated local benchmark instance
298+ (adjust connection and PostgreSQL paths to your environment):
299+
300+ ``` bash
301+ PGPASSWORD=secret pg-perf-bench benchmark \
302+ --connection-type local \
303+ --allow-database-reset \
304+ --host 127.0.0.1 --port 5432 --user postgres \
305+ --database pg_perf_bench_test \
306+ --pg-data-path /var/lib/postgresql/18/main \
307+ --pg-bin-path /usr/lib/postgresql/18/bin \
308+ --workload-profile pagila-htap \
309+ --workload-scale 4 \
310+ --workload-duration-seconds 120 \
311+ --pgbench-clients 1,2,4,8,16 \
312+ --command-timeout 300 \
313+ --report-name pagila-htap-scale4
314+ ```
315+
316+ Use ` --workload-profile pagila ` for the OLTP baseline, or ` imdb ` for analytical
317+ joins. Keep scale, client counts and duration constant when comparing runs.
318+ Scale controls row counts, not a target size in bytes: IMDb scale 1 generates
319+ 100,000 titles and 100,000 people; Pagila scale 1 generates 1,000 films,
320+ 600 customers and 16,000 rentals. Choose data volume relative to the cache
321+ being tested; the profile READMEs describe the datasets in more detail.
322+
323+ The schemas support PostgreSQL 10–18. Generators use deterministic ` hashint8() `
324+ streams; IMDb uses separate streams for related identifiers and attributes to
325+ avoid correlated, degenerate joins. Pagila initializes indexes, identifier
326+ bounds and statistics, and sets ` search_path ` for the database and the benchmark
327+ role within it. No manual role configuration is needed for the supplied commands.
328+ Generated values are reproducible; service timestamps such as Pagila's
329+ ` last_update ` use the current time.
330+
331+ Bundled commands use ` --random-seed=42 ` , and both Pagila variants use
332+ ` -M prepared ` . A fixed seed makes random choices repeatable with the same
333+ client configuration, but a timed run can complete a different number of
334+ scripts; it does not guarantee identical observed mix proportions or TPS.
335+ These profiles use ` profile.json ` , independently of ` pg_workload ` 's
336+ scheduler-specific ` profile.yml ` .
337+
338+ ### Changing script weights or pgbench options
339+
340+ Keep ` --workload-profile ` and supply ` --workload-command ` to replace only its
341+ measured command. The packaged initialization still runs, and
342+ ` ARG_WORKLOAD_PATH ` still points to the packaged profile directory.
343+ ` --init-command ` can similarly replace the initialization command.
344+
345+ For example, append this option to the HTAP command above to change the
346+ reporting weight from 5 to 25, making its target share ` 25 / 125 = 20 % ` :
347+
348+ ``` bash
349+ --workload-command ' ARG_PGBENCH_PATH --no-vacuum --random-seed=42 -M prepared -c ARG_PGBENCH_CLIENTS -j ARG_PGBENCH_CLIENTS -T ARG_WORKLOAD_DURATION_SECONDS -h ARG_PG_HOST -p ARG_PG_PORT -U ARG_PG_USER -f ARG_WORKLOAD_PATH/sql/01_select.sql@50 -f ARG_WORKLOAD_PATH/sql/02_insert.sql@25 -f ARG_WORKLOAD_PATH/sql/03_update.sql@20 -f ARG_WORKLOAD_PATH/sql/04_delete.sql@5 -f ARG_WORKLOAD_PATH/sql/05_reporting.sql@25 ARG_PG_DATABASE'
350+ ```
351+
352+ The replacement is a complete command: include every script you want to run.
353+ Use ` -f FILE@WEIGHT ` for relative weights; omit a file to remove it from the mix.
354+ Set pgbench options such as ` --random-seed ` , ` -M ` and ` -j ` in this command;
355+ there are no separate profile CLI flags for them. Retain the client and duration
356+ placeholders when those values should follow the benchmark settings.
357+
358+ ### Editing SQL or the data generator
359+
360+ Copy the complete profile directory to the workload-generator host and edit
361+ that copy. Using Python from the environment where ` pg-perf-bench ` is installed:
362+
363+ ``` bash
364+ python3 - << 'PY '
365+ from pathlib import Path
366+ from shutil import copytree
367+ import pg_perf_bench
368+
369+ source = Path(pg_perf_bench.__file__).parent / 'workload_profiles' / 'pagila-htap'
370+ copytree(source, 'workloads/pagila-local')
371+ PY
372+ ```
373+
374+ Edit ` generator.py ` to change data distributions. Adjust the probabilities in
375+ ` sql/02_insert.sql ` to change the frequency of customer, catalogue and staff changes.
376+ Edit the SQL scripts for query behavior. The copied ` profile.json ` contains
377+ the command templates, including the script list and weights.
378+
379+ Use ` --benchmark-type custom --workload-path ` for the copy. A custom path does
380+ not automatically apply commands or defaults from ` profile.json ` : pass both command templates explicitly,
381+ and set the duration explicitly if they use ` ARG_WORKLOAD_DURATION_SECONDS ` .
382+ The following reads your edited templates and runs them:
383+
384+ ``` bash
385+ workload_init=$( python3 -c ' import json; print(json.load(open("workloads/pagila-local/profile.json"))["benchmark"]["init_command"])' )
386+ workload_run=$( python3 -c ' import json; print(json.load(open("workloads/pagila-local/profile.json"))["benchmark"]["workload_command"])' )
387+
388+ PGPASSWORD=secret pg-perf-bench benchmark \
389+ --connection-type local \
390+ --allow-database-reset \
391+ --host 127.0.0.1 --port 5432 --user postgres \
392+ --database pg_perf_bench_test \
393+ --pg-data-path /var/lib/postgresql/18/main \
394+ --pg-bin-path /usr/lib/postgresql/18/bin \
395+ --benchmark-type custom \
396+ --workload-path ./workloads/pagila-local \
397+ --workload-scale 4 \
398+ --workload-duration-seconds 120 \
399+ --pgbench-clients 1,2,4,8,16 \
400+ --init-command " $workload_init " \
401+ --workload-command " $workload_run " \
402+ --command-timeout 300 \
403+ --report-name pagila-local-scale4
404+ ```
405+
406+ To validate a configured command without touching PostgreSQL, insert ` plan `
407+ before ` benchmark ` : ` pg-perf-bench plan benchmark ... ` .
408+
409+ ### Profile files and parameters in reports
410+
411+ Both JSON and HTML reports retain the effective initialization and workload
412+ commands for every iteration, together with scale, duration, client counts and
413+ CLI arguments. This includes overridden script weights, seed, jobs and other
414+ pgbench options. Passwords are redacted.
415+
416+ For bundled profiles, ` workload_evidence.files ` contains every file listed in
417+ the manifest and ` profile.json ` itself. For a local profile, the report also
418+ captures the other files under ` --workload-path ` , including JSON, YAML, TOML,
419+ shell scripts and configuration files without extensions. The local manifest's
420+ ` files ` entries supply file roles; they do not select commands or defaults.
421+ Git/Mercurial/Subversion metadata, ` .venv ` , ` venv ` , ` __pycache__ ` , ` .pyc ` and
422+ ` .pyo ` are excluded from automatic traversal. Keep reports, generated datasets
423+ and unrelated files outside the profile directory.
424+
425+ Literal input paths passed to psql or pgbench with ` -f FILE ` , ` -fFILE ` ,
426+ ` --file FILE ` or ` --file=FILE ` are captured too, including external SQL used by
427+ an overridden command and pgbench's ` FILE@WEIGHT ` form. External files are
428+ identified by absolute path in the report; prefer absolute paths in commands.
429+ Relative file arguments are resolved from the directory where the benchmark
430+ was launched. Dependencies opened inside SQL, Python or shell code should be
431+ kept in the local profile directory: arbitrary shell expansion and runtime
432+ dependency discovery are not performed.
433+
434+ Captured files must already exist, be UTF-8 text and be no larger than 5 MiB
435+ each; symlinks are rejected. Their content and SHA-256 are stored in the report
436+ and contribute to both definition and execution hashes. In HTML, open
437+ ** Workload initialization and configuration** for the manifest, schema,
438+ generator and supporting files, and ** pgbench workload** for workload SQL.
439+
440+ ### Managed PostgreSQL
441+
442+ Pass ` --managed-pg-info FILE ` to benchmark an instance whose operating system
443+ and PostgreSQL service are controlled by a cloud provider. The option selects
444+ managed mode automatically; ` --connection-type ` , ` --pg-data-path ` and
445+ ` --pg-bin-path ` are not required. The local ` pgbench ` and ` psql ` clients still
446+ need to be installed.
447+
448+ The metadata file can have ** any format** : JSON, YAML, plain text, PDF, an image
449+ or another binary format. Its format is not parsed or used to configure the
450+ connection. Include the provider, region, instance class, CPU/RAM, storage and
451+ relevant service settings in it. The complete file, its name, size and SHA-256
452+ are embedded as ` managed_pg_info ` in JSON and HTML. The file contents are displayed
453+ directly as ` plain_text ` under ** Managed PostgreSQL instance** . Binary content is
454+ stored and displayed as Base64, with an explicit encoding note.
455+ The file hash also participates in the execution plan and environment identity.
456+
457+ For example:
458+
459+ ``` bash
460+ PGPASSWORD=secret PGSSLMODE=require pg-perf-bench benchmark \
461+ --managed-pg-info ./cloud-instance.yaml \
462+ --host db.example.cloud --port 5432 --user bench_owner \
463+ --database pg_perf_bench_test --allow-database-reset \
464+ --workload-profile pagila-htap --workload-scale 0.1 \
465+ --workload-duration-seconds 30 --pgbench-clients 1,2,4 \
466+ --command-timeout 120 --report-name managed-pagila
467+ ```
468+
469+ Use the TLS settings required by your provider; ` PGSSLMODE ` and ` PGSSLROOTCERT `
470+ apply to the database connections and local client tools.
471+
472+ Managed mode measures TPS, latency, transaction counts and client connection
473+ time, retains the complete workload evidence, and collects PostgreSQL version,
474+ settings and extensions through SQL using the supplied role. It recreates the
475+ dedicated benchmark database before each iteration, so the role needs
476+ ` CREATEDB ` , access to the ` postgres ` maintenance database and ownership of the
477+ benchmark database. ` --allow-database-reset ` remains mandatory.
478+
479+ The utility does not restart the server, flush filesystems, drop OS caches,
480+ install ` postgresql.conf ` , read server logs or run the OS sampler. Host facts,
481+ OS metrics, ` PostgreSQL pg_config ` and server logs explicitly show
482+ ` No data. Managed PostgreSQL. ` These expected limitations have status
483+ ` unsupported ` ; actual SQL or workload failures remain errors. Managed mode
484+ cannot be combined with SSH/Docker transport, ` --pg-custom-config ` or
485+ ` --drop-os-caches ` . ` --collect-pg-logs ` retains the unavailable-data marker.
486+ Custom workload commands execute exactly as supplied and must themselves be
487+ compatible with the provider's permissions.
274488
275489### Iteration lifecycle
276490
277- For each axis value the backend:
491+ For each axis value in the regular mode, the backend:
278492
2794931 . verifies access to the PostgreSQL instance;
2804942 . drops the dedicated benchmark database;
@@ -382,10 +596,12 @@ A benchmark report contains:
382596- a compatibility preflight containing the PostgreSQL server major, the newest
383597 local pgbench/psql versions and the supported server range 10–18;
384598- raw initialization and workload evidence for every completed iteration;
385- - parsed clients, duration, transaction count, average latency, initial
386- connection time, and TPS;
599+ - parsed clients, duration, transaction count, average latency, latency standard
600+ deviation, failed/retried transaction percentages, initial connection time, and TPS;
387601- an explicit ` maximum_tps ` point with its axis value and complete metrics;
388- - a TPS chart for the selected axis;
602+ - separate charts for TPS, average transaction latency, transaction latency
603+ standard deviation, failed/retried transaction percentages and initial
604+ connection time, all using the selected client or duration axis;
389605- all ` pg_diag ` OS charts collected during every measured iteration: CPU
390606 utilization/load, RAM usage/pressure, disk throughput/IOPS/utilization/latency,
391607 and network throughput/packets;
@@ -395,6 +611,14 @@ A benchmark report contains:
395611- an optional PostgreSQL log archive reference backed by the report-local
396612 ` db_logs/ ` directory.
397613
614+ Optional metrics are taken from the overall pgbench summary. Percentages retain
615+ the values reported by pgbench. Retry statistics require ` --max-tries ` other than
616+ 1; latency standard deviation requires timing statistics (for example,
617+ ` --progress ` ); ` --connect ` reports average connection time instead of initial
618+ connection time. Absent values remain ` null ` , appear as gaps, and display an
619+ explicit no-data message when an entire chart is unavailable. A reported zero
620+ remains a zero. Joined reports include a series per source for these charts.
621+
398622The JSON and HTML files are written through temporary files and atomically
399623renamed into place. Report names cannot contain path separators, ` . ` /` .. ` , or a
400624NUL byte.
0 commit comments