feat: report storage snapshots and total benchmark timing

Capture all database sizes and top 100 tables and indexes before and after each workload. Wait for OS sampling to finish before the after snapshot so size queries do not contaminate the final samples.

Preserve storage evidence in JOIN reports, show start and finish timestamps and total duration, expand key report items, and document the collection sequence. Keep version 0.4.0.

Author: O2eg

README.md

@@ -240,7 +240,7 @@ PGPASSWORD=secret pg-perf-bench benchmark \
   --report-name local-pg18
 ```
 
-The command timeout applies independently to initialization and workload
+The command timeout applies independently to initialization, `VACUUM ANALYZE`, and workload
 commands. It must be longer than the expected command duration.
 
 ### Workload placeholders
@@ -288,7 +288,7 @@ several SQL statements or transactions. The default HTAP reporting share is
 | Data volume | `--workload-scale SCALE`, default `1`; positive fractional values such as `0.25` are accepted. Generators retain minimum table sizes at small scales. |
 | Concurrent clients | `--pgbench-clients 1,2,4,8,16`; each value gets a freshly recreated database. Bundled commands also use one pgbench job per client. |
 | Measured window per point | `--workload-duration-seconds 120`; overrides the profile default for every client count. |
-| Command time limit | `--command-timeout 300`; allow enough time for initialization and for the workload window plus completion of in-flight queries. |
+| Command time limit | `--command-timeout 300`; allow enough time for initialization, pre-workload `VACUUM ANALYZE`, and the workload window plus completion of in-flight queries. |
 
 Bundled profiles require `--pgbench-clients`; `--pgbench-time` is rejected.
 They select benchmark type `custom` automatically. `--workload-path` cannot be
@@ -496,9 +496,11 @@ For each axis value in the regular mode without Patroni, the backend:
 4. flushes filesystems and optionally drops host OS caches;
 5. starts PostgreSQL and recreates the database;
 6. runs the initialization command;
-7. runs the workload command while the `pg_diag` Linux sampler records CPU,
+7. runs `VACUUM ANALYZE` and captures the Before workload size snapshot;
+8. runs the workload command while the `pg_diag` Linux sampler records CPU,
    RAM, disk and network metrics on the database host;
-8. stores raw stdout, stderr, return code, UTC start time, elapsed time, parsed
+9. waits for any remaining OS sampling to finish, then captures the After workload size snapshot;
+10. stores both snapshots, raw stdout, stderr, return code, UTC start time, elapsed time, parsed
    pgbench metrics, and iteration metadata.
 
 After the final iteration it collects the configured host and PostgreSQL facts
@@ -645,6 +647,21 @@ required on the SSH server.
 
 ## Report contents
 
+Below `parameters`, the report header shows `Started:`, `Finished:`, and
+`Common duration:` on separate lines. New reports record UTC timestamps and a
+total elapsed duration in `timing`. For benchmarks, this covers preparation,
+every workload iteration, and final data/log collection, up to report
+serialization. Collection and JOIN reports measure their own operations.
+Duration is measured with a monotonic clock and displayed as `HH:MM:SS.mmm`.
+Older reports retain their recorded start time; missing finish times and total
+durations display `Not recorded`.
+
+The main measurements, TPS, average latency and failed-transaction charts,
+database version/settings, replication policy/settings/slots/senders, storage
+snapshots, pgbench options, CPU/RAM capacity, disk space, and key CPU/memory/disk
+timelines are expanded by default. Detailed source code and secondary items
+remain collapsible; `Expand all` and `Collapse all` still control the full report.
+
 A benchmark report contains:
 
 - artifact schema and generator versions;
@@ -658,6 +675,8 @@ A benchmark report contains:
 - a compatibility preflight containing the PostgreSQL server major, the newest
   local pgbench/psql versions and the supported server range 10–18;
 - raw initialization and workload evidence for every completed iteration;
+- all database sizes and the workload database's top 100 tables/indexes, captured
+  before and after every measured workload;
 - parsed clients, duration, transaction count, average latency, latency standard
   deviation, failed/retried transaction percentages, initial connection time, and TPS;
 - an explicit `maximum_tps` point with its axis value and complete metrics;
@@ -695,6 +714,49 @@ pg-perf-bench render \
   --out report/local-pg18.html
 ```
 
+### Storage sizes before and after workload
+
+Benchmark reports include **Storage sizes before and after workload**, with six
+items for **every iteration**:
+
+| Snapshot | Items |
+| --- | --- |
+| Before workload | All database sizes; top 100 tables by total size; top 100 indexes by size |
+| After workload | All database sizes; top 100 tables by total size; top 100 indexes by size |
+
+The order is: recreate the workload database → run `init_command` → run
+`VACUUM ANALYZE` → collect **Before workload** → run `pgbench` → finish any
+remaining OS sampling → collect **After workload**. Preparation and size collection
+are outside the measured pgbench command. Waiting for the OS sampler prevents
+size-query CPU and I/O from entering its final samples. An explicit
+`--system-metrics-duration` longer than the workload delays the after snapshot
+until sampling finishes. No additional vacuum is run before the after snapshot.
+`VACUUM ANALYZE` uses `--command-timeout`; a failure stops the benchmark before
+the workload starts.
+
+The database item lists **every database on the instance**, including templates,
+with `is_workload_database` marking the target. Sizes cover database files across
+tablespaces; WAL and shared cluster files are excluded. An inaccessible database
+remains listed with a null size and its collection error. Other databases' sizes
+are retained.
+
+Table and index items cover the **workload database**. They use measured sizes,
+not catalog page estimates, and sort before applying the 100-row limit. Tables
+are ranked by total size including indexes and TOAST; the table, index and TOAST
+sizes are also shown separately. TOAST is already included in the table size.
+Stored leaf partitions and materialized views are included; partitioned parents
+without storage and system objects are omitted. Index sizes include all forks;
+TOAST indexes are accounted for in the table item. Sizes are integer bytes with
+human-readable totals.
+
+Every snapshot records its collection interval in JSON and HTML. Measurements
+are sequential, so concurrent activity can change sizes during collection.
+SQL statements have a 10-second timeout; errors remain explicit in the report.
+Snapshots are retained in `benchmark_runs[].storage` before the next iteration
+recreates the database. JOIN preserves all source snapshots and identifies older
+reports that did not collect them. PostgreSQL 10–18 and managed PostgreSQL use
+the same SQL collection path.
+
 ### Replication evidence
 
 Benchmark, `collect-db-info`, and `collect-all-info` reports include a

doc/logic_building_and_comparing_reports.md

@@ -103,6 +103,11 @@ Join requires at least two source reports with:
 - complete chart and pgbench-result structures;
 - every required join-task path present and equal.
 
+The Storage section preserves each iteration's Before workload and After workload
+items from every source, including database sizes and top tables/indexes. Sources
+without storage snapshots remain joinable and receive an explicit missing-evidence
+item. Required storage paths in a JOIN task must still exist and match.
+
 The Replication section is optional for older `report-v1` sources. A mixed join
 preserves each source's replication snapshot separately and labels sources that
 did not collect it. Missing replication evidence is not an empty slot list or a

doc/workload_description.md

@@ -18,6 +18,14 @@ axis value is used.
 Every axis value is a separate destructive iteration with a freshly recreated
 database.
 
+After `init_command`, the runner executes `VACUUM ANALYZE` and collects the
+**Before workload** size snapshot. It then runs the measured command, waits for
+any remaining OS sampling to finish, and collects **After workload**, without
+another vacuum. This keeps size-query CPU and I/O out of the OS samples. Each snapshot
+lists all databases on the instance and the workload database's top 100 tables
+and indexes by measured size. Snapshots are retained separately for every axis
+value. See [storage report contents](../README.md#storage-sizes-before-and-after-workload).
+
 Bundled profiles intentionally use `--pgbench-clients`: they sweep concurrency
 to find maximum TPS for one fixed dataset, query mix and environment.
 
@@ -96,6 +104,7 @@ recreates the database for each concurrency point and measures maximum TPS.
 `--command-timeout` is a positive number of seconds applied independently to:
 
 - every initialization command;
+- the pre-workload `VACUUM ANALYZE`;
 - every measured workload command;
 - transport host commands unless an item defines a smaller timeout.
 

src/pg_perf_bench/benchmark.py

@@ -3,8 +3,10 @@
 import platform
 import re
 import sys
+import time
 from contextlib import nullcontext
 from copy import deepcopy
+from datetime import datetime, timezone
 from pathlib import Path
 from typing import Any
 
@@ -43,6 +45,11 @@
 from pg_perf_bench.pgbench_metrics import LEGACY_METRIC_KEYS, parse_pgbench_metrics
 from pg_perf_bench.report.commands import fill_info_report
 from pg_perf_bench.report.processing import get_report_structure
+from pg_perf_bench.storage import (
+    build_storage_section,
+    collect_storage_snapshot,
+    vacuum_analyze,
+)
 from pg_perf_bench.system_metrics import (
     build_system_metrics_section,
     collect_system_metrics,
@@ -328,6 +335,9 @@ async def run_benchmark_with_evidence(
             env=environment,
             secrets=secrets,
         )
+        await vacuum_analyze(logger, db_conf, command_timeout)
+        logger.info('Collecting storage sizes before workload.')
+        storage_before = await collect_storage_snapshot(logger, db_conf)
         logger.info('Executing pgbench workload command.')
         sampler_task = None
         if connection is not None and connection_type != ConnectionType.MANAGED:
@@ -353,7 +363,11 @@ async def run_benchmark_with_evidence(
                 env=environment,
                 secrets=secrets,
             )
+            # Providers can finish after pgbench. Keep size-query CPU and I/O
+            # outside their final sampling interval.
             system_metrics = await sampler_task if sampler_task is not None else None
+            logger.info('Collecting storage sizes after workload.')
+            storage_after = await collect_storage_snapshot(logger, db_conf)
         except BaseException:
             if sampler_task is not None and not sampler_task.done():
                 sampler_task.cancel()
@@ -370,6 +384,7 @@ async def run_benchmark_with_evidence(
             'workload': workload_result.as_dict(secrets=secrets),
             'metrics': metrics,
             'legacy_metrics': [metrics[key] for key in LEGACY_METRIC_KEYS],
+            'storage': {'before_workload': storage_before, 'after_workload': storage_after},
         }
         if system_metrics is not None:
             result['system_metrics'] = system_metrics
@@ -589,6 +604,8 @@ async def run_benchmark_and_collect_metrics(
         """
         Main entry point to execute the full benchmarking workflow.
         """
+        started_at = datetime.now(timezone.utc).isoformat()
+        started_clock = time.monotonic()
         display_user_configuration(args, logger)
 
         try:
@@ -632,6 +649,8 @@ async def run_benchmark_and_collect_metrics(
                 ),
                 'workload_definition_hash': workload_evidence['definition_hash'],
                 'workload_execution_hash': workload_evidence['execution_hash'],
+                'vacuum_analyze_before_each_workload': True,
+                'storage_snapshots': ['before_workload', 'after_workload'],
                 'system_metrics_engine': None if managed_path else 'pg_diag',
                 'system_metrics_collected_during_workload': not bool(managed_path),
                 'system_metrics_interval_seconds': (
@@ -672,6 +691,7 @@ async def run_benchmark_and_collect_metrics(
                 report_data['pgbench_outputs'] = [run['legacy_metrics'] for run in benchmark_runs]
                 report['benchmark_runs'] = benchmark_runs
                 report['maximum_tps'] = BenchmarkRunner.maximum_tps(benchmark_runs)
+                report['sections']['storage'] = build_storage_section(benchmark_runs)
                 report['sections']['os_metrics'] = build_system_metrics_section(benchmark_runs)
                 if managed_path:
                     section = report['sections']['os_metrics']
@@ -702,6 +722,11 @@ async def run_benchmark_and_collect_metrics(
                 report['database_configuration_evidence'] = database_evidence
 
             logger.info('Benchmarking process completed successfully.')
+            report['timing'] = {
+                'started_at': started_at,
+                'finished_at': datetime.now(timezone.utc).isoformat(),
+                'elapsed_seconds': time.monotonic() - started_clock,
+            }
             return report
 
         except Exception as e:

src/pg_perf_bench/collect_info.py

@@ -1,5 +1,7 @@
 import platform
 import sys
+import time
+from datetime import datetime, timezone
 
 import asyncpg
 
@@ -113,6 +115,8 @@ async def collect_info(
         Orchestrates full process of collecting system and DB info,
         generating and returning a report dictionary.
         """
+        started_at = datetime.now(timezone.utc).isoformat()
+        started_clock = time.monotonic()
         if 'report_template' not in report_conf:
             logger.error('Missing "report_template" in report_conf.')
             return None
@@ -165,6 +169,11 @@ async def collect_info(
                     logger.info('Database connection closed.')
 
             logger.info('Collect info process completed successfully.')
+            report['timing'] = {
+                'started_at': started_at,
+                'finished_at': datetime.now(timezone.utc).isoformat(),
+                'elapsed_seconds': time.monotonic() - started_clock,
+            }
             return report
 
         except FileNotFoundError as fe:

src/pg_perf_bench/commands/sql_commands/storage_indexes.sql

@@ -0,0 +1,24 @@
+-- Include newly grown indexes even if their catalog statistics have not been refreshed.
+WITH sizes AS (
+    SELECT idx.oid::bigint AS index_oid,
+           n.nspname::text AS schema_name,
+           tbl.relname::text AS table_name,
+           idx.relname::text AS index_name,
+           am.amname::text AS access_method,
+           i.indisprimary AS is_primary,
+           i.indisvalid AS is_valid,
+           pg_catalog.pg_table_size(idx.oid) AS index_size_bytes
+    FROM pg_catalog.pg_index i
+    JOIN pg_catalog.pg_class idx ON idx.oid = i.indexrelid
+    JOIN pg_catalog.pg_class tbl ON tbl.oid = i.indrelid
+    JOIN pg_catalog.pg_namespace n ON n.oid = idx.relnamespace
+    JOIN pg_catalog.pg_am am ON am.oid = idx.relam
+    WHERE idx.relkind = 'i' AND tbl.relkind IN ('r', 'm')
+      AND n.nspname NOT IN ('pg_catalog', 'information_schema')
+      AND n.nspname !~ '^pg_(toast|temp)_'
+      AND n.nspname <> 'pg_toast'
+)
+SELECT *, pg_catalog.pg_size_pretty(index_size_bytes) AS index_size
+FROM sizes
+ORDER BY index_size_bytes DESC NULLS LAST, schema_name, table_name, index_name, index_oid
+LIMIT 100;

src/pg_perf_bench/commands/sql_commands/storage_tables.sql

@@ -0,0 +1,24 @@
+-- Measure every stored user table before limiting: relpages can be stale after the workload.
+WITH sizes AS (
+    SELECT c.oid::bigint AS table_oid,
+           n.nspname::text AS schema_name,
+           c.relname::text AS table_name,
+           c.relkind::text AS relation_kind,
+           c.relispartition AS is_partition,
+           pg_catalog.pg_table_size(c.oid) AS table_size_bytes,
+           pg_catalog.pg_indexes_size(c.oid) AS indexes_size_bytes,
+           CASE WHEN c.reltoastrelid <> 0
+                THEN pg_catalog.pg_total_relation_size(c.reltoastrelid)
+                ELSE 0::bigint END AS toast_size_bytes,
+           pg_catalog.pg_total_relation_size(c.oid) AS total_size_bytes
+    FROM pg_catalog.pg_class c
+    JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace
+    WHERE c.relkind IN ('r', 'm')
+      AND n.nspname NOT IN ('pg_catalog', 'information_schema')
+      AND n.nspname !~ '^pg_(toast|temp)_'
+      AND n.nspname <> 'pg_toast'
+)
+SELECT *, pg_catalog.pg_size_pretty(total_size_bytes) AS total_size
+FROM sizes
+ORDER BY total_size_bytes DESC NULLS LAST, schema_name, table_name, table_oid
+LIMIT 100;