feat: release v0.7.0 with corrected workloads and direct SQL transport

Use direct database connections with SSH reserved for host operations and
metrics. Correct Pagila and IMDb data distributions, temporal semantics,
query aggregates and selective indexes, and document profile size estimates.

Return newly created Pagila rentals through the open-inventory index and
serialize inventory moves with rental creation using a fresh snapshot after
the row lock. Protect JSON strings during legacy lshw structural recovery.
Add regression coverage for rental reuse, concurrent moves and JSON values.

Validation: 363 unit tests and 10 PostgreSQL 18.6 Pagila integration tests
passed; Ruff, wheel/sdist metadata and documentation checks, and installed
wheel validation passed. Chart tests require Node.js and were skipped locally.

Author: Gurov Oleg

README.md

@@ -1,5 +1,7 @@
 # pg-perf-bench
 
+For this benchmark stand, see the [source synchronization workflow](doc/sidecar-source-workflow.md).
+
 ## Overview
 
 Based on [pg_perfbench](https://github.com/TantorLabs/pg_perfbench).
@@ -65,7 +67,7 @@ selected target. The workload commands themselves run on the machine where
 |---|---|---|
 | `local` | local machine | local machine |
 | `docker` | existing container | local machine through a published port |
-| `ssh` | remote host | local machine through an SSH local-forwarding port |
+| `ssh` | remote host | local machine, direct TCP to PostgreSQL or a pooler |
 | `--managed` | PostgreSQL protocol only; no host transport | local machine through the managed endpoint |
 
 This separation keeps workload generation independent of target management and
@@ -229,6 +231,12 @@ collects host inventory without sudo and keeps the target container's
 `pg_config` evidence separate. Raw interface state is retained, but runtime
 Docker bridges do not participate in the stable JOIN hardware identity.
 
+For legacy `lshw 02.18.x` (including Debian 11), hardware tables recover the
+known malformed `-class ... -json` output using the same narrow repairs as
+pg-diag. The fallback runs only after strict JSON parsing fails and only for
+`lshw_*.sh` collectors. An absent hardware class reported as `]` becomes an
+empty table; unrecoverable output still produces a collection error.
+
 ## Running a benchmark
 
 Exactly one iteration axis is required:
@@ -701,10 +709,8 @@ The workload reaches PostgreSQL through the port published on `--host` and
 --ssh-user postgres \
 --ssh-key /secure/path/id_ed25519 \
 --ssh-known-hosts /secure/path/known_hosts \
---remote-pg-host 127.0.0.1 \
---remote-pg-port 5432 \
---host 127.0.0.1 \
---port 55432
+--host db-host.example \
+--port 5432
 ```
 
 To use an identity already loaded into a local agent, replace `--ssh-key` with
@@ -721,10 +727,8 @@ pg-perf-bench collect-all-info \
   --ssh-user postgres \
   --ssh-agent \
   --ssh-known-hosts /secure/path/known_hosts \
-  --remote-pg-host 127.0.0.1 \
-  --remote-pg-port 5432 \
-  --host 127.0.0.1 \
-  --port 55432 \
+  --host db-host.example \
+  --port 5432 \
   --database appdb \
   --pg-bin-path /usr/lib/postgresql/18/bin
 ```
@@ -736,11 +740,15 @@ agent, run `ssh-add`, or forward the agent to the remote host. Both modes use
 explicit public-key authentication and the selected `known_hosts` policy
 without loading the user's OpenSSH configuration.
 
-For database modes, `--host` and `--port` are the local bind address and
-free port. `--remote-pg-host` and `--remote-pg-port` identify PostgreSQL from
-the SSH server. Host commands run remotely; `asyncpg`, `pgbench`, and `psql`
-connect through native AsyncSSH local forwarding. No `AcceptEnv` change is
-required on the SSH server.
+For database modes, `--host` and `--port` are the PostgreSQL or pooler endpoint
+reachable directly from the load-generator host. `asyncpg`, `pgbench`, and `psql`
+connect to this endpoint directly. `--ssh-host` selects the host for OS metrics,
+host commands and lifecycle operations; it can differ from the SQL endpoint.
+AsyncSSH does not forward database traffic. No `AcceptEnv` change is required.
+
+Older commands using `--remote-pg-host` or `--remote-pg-port` fail with a migration
+message before connecting. Remove those options and replace the former local
+bind address/port with the directly reachable database endpoint.
 
 ## Report contents
 

doc/sidecar-source-workflow.md

@@ -0,0 +1,37 @@
+# Working from source on a benchmark sidecar
+
+Keep one authoritative Git checkout, make and review changes there, then synchronize
+it to the load-generator host. SQL connects directly to PostgreSQL or its session
+pool through `--host` and `--port`; SSH is used for host operations and OS metrics.
+
+The repository helper `scripts/sync_sidecar.py` mirrors the checkout to
+`~/pg_perf_bench` on the selected sidecar. Supply the destination and SSH key
+explicitly when using it on your own infrastructure:
+
+```bash
+python3 scripts/sync_sidecar.py --host bench@sidecar.example --key /path/to/key --dry-run
+python3 scripts/sync_sidecar.py --host bench@sidecar.example --key /path/to/key
+```
+
+The helper expects an existing `~/pgpb-venv` Python 3.12 environment with the
+project dependencies and CLI installed. Dependency changes require a separate
+environment update; synchronization does not install dependencies.
+
+Do not synchronize while a benchmark is active. The helper checks for an active
+CLI benchmark before changing files. A dry run only previews the rsync changes.
+Before a real synchronization it backs up the remote source and source-path
+configuration under `~/bench/pgpb_source_sync_<UTC timestamp>/`.
+
+The mirror includes removal of files that no longer exist in the checkout.
+Git data, virtual environments, caches, local `.env` files and build/runtime
+outputs are excluded. Benchmark results under `~/bench` are outside the mirror.
+
+The existing `~/pgpb-venv/bin/pg-perf-bench` command imports directly from
+`~/pg_perf_bench/src` through the environment's `pg_perf_bench_source.pth` file.
+SQL, templates and other resources are read from that source tree. No wheel build
+or package reinstall is required. The helper checks the resulting import path
+and runs `pg-perf-bench validate`; start a fresh process to use the updated code.
+
+To roll back, restore the saved `source/` directory and `.pth` configuration
+identified by the selected backup's `state.json`. If `pth_existed` is false,
+remove only `pg_perf_bench_source.pth` to return to the installed package.

doc/ssh_mode_usage.md

@@ -2,14 +2,15 @@
 
 Use SSH transport when PostgreSQL and host fact collection run on a remote
 server while `pg_perf_bench`, `pgbench`, and `psql` run locally. The transport
-uses AsyncSSH and native local port forwarding; `sshtunnel` and server-side
+uses AsyncSSH for remote host operations. SQL connects directly over TCP from
+the load-generator host; there is no database port forwarding. Server-side
 `AcceptEnv` changes are not required.
 
 ## Execution boundary
 
 | Operation | Location |
 |---|---|
-| PostgreSQL connection | local forwarded port |
+| PostgreSQL connection | direct TCP from the load generator to `--host:--port` |
 | `pgbench` and `psql` | local workload-generator host |
 | Host fact collectors | remote host |
 | Timed `pg_diag` OS sampler | remote host |
@@ -50,14 +51,23 @@ disposable stands.
 
 ## Address model
 
-SSH database modes use two address pairs:
+SSH mode separates the SQL endpoint from the host-management endpoint:
 
-- `--host` and `--port` define a free local bind address and port;
-- `--remote-pg-host` and `--remote-pg-port` identify PostgreSQL as seen from
-  the SSH server.
+- `--host` and `--port` identify PostgreSQL or a pooler reachable directly from
+  the machine running `pg-perf-bench`;
+- `--ssh-host` and `--ssh-port` identify the database host used for OS metrics,
+  fact collection and lifecycle operations.
 
-`asyncpg`, `pgbench`, and `psql` connect to the local pair. AsyncSSH forwards
-that traffic to the remote pair. The local port must not already be in use.
+`asyncpg`, `pgbench`, and `psql` use the SQL endpoint directly. This keeps SSH
+forwarding and encryption out of the measured database traffic. The endpoints
+can differ, for example when SQL goes through HAProxy/PgBouncer while metrics
+are collected on the primary. Select the SSH host corresponding to the database
+server whose OS metrics you intend to measure.
+
+The former `--remote-pg-host` and `--remote-pg-port` options are rejected before
+any connection is made. To migrate, remove both options and replace the old
+local bind address and port with the SQL endpoint reachable from the generator.
+A working SSH login alone does not establish database network access.
 
 ## Read-only collection
 
@@ -68,18 +78,16 @@ PGPASSWORD=secret pg-perf-bench collect-all-info \
   --ssh-user postgres \
   --ssh-key ~/.ssh/pg_perf_bench \
   --ssh-known-hosts ~/.ssh/known_hosts \
-  --remote-pg-host 127.0.0.1 \
-  --remote-pg-port 5432 \
-  --host 127.0.0.1 \
-  --port 55432 \
+  --host db-host.example \
+  --port 5432 \
   --user postgres \
   --database postgres \
   --pg-bin-path /usr/lib/postgresql/18/bin \
   --report-name ssh-facts
 ```
 
-Collection opens the tunnel and runs remote fact commands, but does not stop
-PostgreSQL or install a configuration file.
+Collection opens a direct SQL connection and runs host fact commands over SSH.
+It does not stop PostgreSQL or install a configuration file.
 
 ## Benchmark
 
@@ -91,10 +99,8 @@ PGPASSWORD=secret pg-perf-bench benchmark \
   --ssh-user postgres \
   --ssh-key ~/.ssh/pg_perf_bench \
   --ssh-known-hosts ~/.ssh/known_hosts \
-  --remote-pg-host 127.0.0.1 \
-  --remote-pg-port 5432 \
-  --host 127.0.0.1 \
-  --port 55432 \
+  --host db-host.example \
+  --port 5432 \
   --user postgres \
   --database pg_perf_bench_test \
   --pg-data-path /var/lib/postgresql/18/main \
@@ -120,8 +126,7 @@ and use a bundled profile instead:
 PGPASSWORD=secret pg-perf-bench benchmark \
   --connection-type ssh --ssh-host db-host.example --ssh-user postgres \
   --ssh-key ~/.ssh/pg_perf_bench --ssh-known-hosts ~/.ssh/known_hosts \
-  --remote-pg-host 127.0.0.1 --remote-pg-port 5432 \
-  --host 127.0.0.1 --port 55432 --user postgres \
+  --host db-host.example --port 5432 --user postgres \
   --pg-data-path /var/lib/postgresql/18/main \
   --pg-bin-path /usr/lib/postgresql/18/bin \
   --database pg_perf_bench_test --allow-database-reset --reset-mode schema \
@@ -163,7 +168,7 @@ and atomically renamed to the remote cluster's `postgresql.conf` before the rese
 sudo rule. Hardware collectors use `sudo -n` and fail fast when permission is
 not available.
 
-The load commands always execute locally through the tunnel. The newest local
+The load commands always execute locally using the direct SQL endpoint. The newest local
 pgbench is selected automatically even when the remote server is PostgreSQL
 10–18. Concurrent CPU, RAM, disk, and network sampling executes on the remote
 database host through the already authenticated SSH session.
@@ -176,7 +181,6 @@ database host through the already authenticated SSH session.
   permissions, and the installed public key;
 - authentication failure in agent mode: verify `SSH_AUTH_SOCK`, `ssh-add -l`,
   the agent lifetime, and the installed public key;
-- local bind failure: choose an unused `--port`;
-- PostgreSQL connection failure with working SSH: verify the remote address,
-  PostgreSQL authentication, and listen rules;
+- PostgreSQL connection failure with working SSH: verify reachability of
+  `--host:--port` from the generator, PostgreSQL authentication and listen rules;
 - lifecycle failure: ensure `--ssh-user` owns or can control the cluster.

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "pg-perf-bench"
-version = "0.6.1"
+version = "0.7.0"
 description = "Reproducible PostgreSQL TPS benchmarks with environment evidence and report comparison"
 readme = "README.md"
 requires-python = ">=3.10"

scripts/sync_sidecar.py

@@ -0,0 +1,95 @@
+#!/usr/bin/env python3
+"""Sync this source checkout to the benchmark sidecar without building a package."""
+
+import argparse
+import shlex
+import subprocess
+from pathlib import Path
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument('--host', default='psql-admin@10.234.9.110')
+    parser.add_argument('--key', type=Path, default=Path('/home/oleg/Desktop/access/id_ed25519'))
+    parser.add_argument('--dry-run', action='store_true')
+    args = parser.parse_args()
+    if args.host.startswith('-') or any(c.isspace() for c in args.host):
+        parser.error('host must be a single SSH destination')
+    source = Path(__file__).resolve().parents[1]
+    assert (source / 'src/pg_perf_bench/cli.py').is_file()
+    assert (source / 'pyproject.toml').is_file()
+    ssh = ['ssh', '-i', str(args.key), '-o', 'BatchMode=yes']
+
+    def remote(code):
+        result = subprocess.run(
+            [*ssh, args.host, '~/pgpb-venv/bin/python -'],
+            input=code,
+            text=True,
+            capture_output=True,
+            check=True,
+        )
+        print(result.stdout, end='', flush=True)
+        return result.stdout
+
+    if not args.dry_run:
+        remote("""from pathlib import Path
+import datetime, json, shutil
+h=Path.home()
+for proc in Path('/proc').glob('[0-9]*/cmdline'):
+    try:
+        argv=proc.read_bytes().decode(errors='replace').split('\\0')
+    except (OSError, ProcessLookupError):
+        continue
+    if 'benchmark' in argv and any(Path(a).name in ('pg-perf-bench','pg_perf_bench') for a in argv):
+        raise SystemExit('A benchmark is running; sync after it completes.')
+backup=h/'bench'/('pgpb_source_sync_'+datetime.datetime.now(datetime.timezone.utc).strftime('%Y%m%dT%H%M%S%fZ'))
+backup.mkdir(parents=True)
+source=h/'pg_perf_bench'
+if source.exists(): shutil.copytree(source,backup/'source',symlinks=True)
+pth=h/'pgpb-venv/lib/python3.12/site-packages/pg_perf_bench_source.pth'
+if pth.exists(): shutil.copy2(pth,backup/pth.name)
+(backup/'state.json').write_text(json.dumps({'source':str(source),'pth':str(pth),'pth_existed':pth.exists()},indent=2)+'\\n')
+print('Backup:',backup)
+""")
+    cmd = ['rsync', '-a', '--checksum', '--delete-delay', '--itemize-changes']
+    for pattern in (
+        '.git',
+        '__pycache__',
+        '*.py[cod]',
+        '*.egg-info',
+        '.pytest_cache',
+        '.ruff_cache',
+        '/.venv/',
+        '/venv/',
+        '/build/',
+        '/dist/',
+        '/log/',
+        '/report/',
+        '/db_logs/',
+        '/.env',
+        '/.env.*',
+    ):
+        cmd.extend(['--exclude', pattern])
+    if args.dry_run:
+        cmd.append('--dry-run')
+    cmd.extend(['-e', shlex.join(ssh), '--', str(source) + '/', args.host + ':pg_perf_bench/'])
+    subprocess.run(cmd, check=True)
+    if args.dry_run:
+        return
+    remote("""from pathlib import Path
+import json, subprocess
+h=Path.home();source=h/'pg_perf_bench/src'
+pth=h/'pgpb-venv/lib/python3.12/site-packages/pg_perf_bench_source.pth'
+pth.write_text('import sys; sys.path.insert(0, '+repr(str(source))+')\\n')
+code=("import pg_perf_bench; from pathlib import Path; "
+      "p=Path(pg_perf_bench.__file__).resolve(); "
+      "assert p == Path.home()/'pg_perf_bench/src/pg_perf_bench/__init__.py', p; "
+      "print('Active source:',p)")
+subprocess.run([str(h/'pgpb-venv/bin/python'),'-c',code],cwd='/tmp',check=True)
+subprocess.run([str(h/'pgpb-venv/bin/pg-perf-bench'),'validate'],cwd='/tmp',check=True)
+""")
+    print('Source synchronized; existing venv dependencies retained. No pip build/install.')
+
+
+if __name__ == '__main__':
+    main()

src/pg_perf_bench/__init__.py

@@ -1,3 +1,3 @@
 """PostgreSQL performance benchmark and evidence collector."""
 
-__version__ = '0.6.1'
+__version__ = '0.7.0'

src/pg_perf_bench/cli.py

@@ -110,7 +110,7 @@ def _add_host_args(parser: argparse.ArgumentParser, *, required: bool = True) ->
     parser.add_argument('--pg-data-path')
     parser.add_argument('--pg-bin-path')
     parser.add_argument('--container-name')
-    parser.add_argument('--ssh-host')
+    parser.add_argument('--ssh-host', help='remote host for OS metrics and host operations')
     parser.add_argument('--ssh-port', type=positive_int, default=22)
     parser.add_argument('--ssh-user', default='postgres')
     ssh_auth = parser.add_mutually_exclusive_group()
@@ -122,17 +122,28 @@ def _add_host_args(parser: argparse.ArgumentParser, *, required: bool = True) ->
     )
     parser.add_argument('--ssh-known-hosts')
     parser.add_argument('--ssh-insecure-no-host-key-check', action='store_true')
-    parser.add_argument('--remote-pg-host')
-    parser.add_argument('--remote-pg-port', type=positive_int)
+    parser.add_argument('--remote-pg-host', help=argparse.SUPPRESS)
+    parser.add_argument('--remote-pg-port', type=positive_int, help=argparse.SUPPRESS)
 
 
 def _add_database_args(
     parser: argparse.ArgumentParser,
     *,
     include_custom_config: bool = False,
 ) -> None:
-    parser.add_argument('--host', '--pg-host', dest='pg_host')
-    parser.add_argument('--port', '--pg-port', dest='pg_port', type=positive_int)
+    parser.add_argument(
+        '--host',
+        '--pg-host',
+        dest='pg_host',
+        help='PostgreSQL or pooler host reachable directly from the load generator',
+    )
+    parser.add_argument(
+        '--port',
+        '--pg-port',
+        dest='pg_port',
+        type=positive_int,
+        help='direct PostgreSQL or pooler TCP port',
+    )
     parser.add_argument('--user', '--pg-user', dest='pg_user', default='postgres')
     parser.add_argument(
         '--password',